-- Logs begin at Thu 2022-08-11 10:09:08 UTC, end at Thu 2022-08-11 11:22:03 UTC. -- Aug 11 10:41:53.496072 dv-cinder-820027-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 11 10:41:53.504765 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 11 10:41:53 2022] *** Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: nodename: dv-cinder-820027-17-iscsi Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: machine: x86_64 Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: clock source: unix Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: pcre jit disabled Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: detected number of CPU cores: 4 Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: current working directory: / Aug 11 10:41:53.507005 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: detected binary path: /usr/bin/uwsgi-core Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: your processes number limit is 47680 Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: your memory page size is 4096 bytes Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: detected max file descriptor number: 2048 Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: lock engine: pthread robust mutexes Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: thunder lock: enabled Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 11 10:41:53.507597 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 11 10:41:53.546478 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: Python main interpreter initialized at 0x5580d08d79c0 Aug 11 10:41:53.546478 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: python threads support enabled Aug 11 10:41:53.546478 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: your server socket listen backlog is limited to 100 connections Aug 11 10:41:53.546478 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: your mercy for graceful operations on workers is 90 seconds Aug 11 10:41:53.546830 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: mapped 403077 bytes (393 KB) for 2 cores Aug 11 10:41:53.547073 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: *** Operational MODE: preforking *** Aug 11 10:41:53.547310 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: *** uWSGI is running in multiple interpreter mode *** Aug 11 10:41:53.547310 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: spawned uWSGI master process (pid: 81563) Aug 11 10:41:53.547495 dv-cinder-820027-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 11 10:41:53.548884 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 1 (pid: 81570, cores: 1) Aug 11 10:41:53.548884 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 2 (pid: 81571, cores: 1) Aug 11 10:41:53.548884 dv-cinder-820027-17-iscsi devstack@keystone.service[81563]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 11 10:41:59.023956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 10:41:59.023956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 11 10:41:59.024744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 10:41:59.024744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Aug 11 10:41:59.025182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 10:41:59.025182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 11 10:41:59.025314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 10:41:59.025314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Aug 11 10:41:59.073587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 10:41:59.073752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 10:41:59.073838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 10:41:59.073928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 10:41:59.074014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] command line args: [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 10:41:59.074096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] command line args: [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 10:41:59.074267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 10:41:59.074357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 10:41:59.074429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ================================================================================ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 10:41:59.074500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ================================================================================ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 10:41:59.074682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.074762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.074925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_dir = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_dir = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_source = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_source = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] debug = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] debug = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.075643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.076937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_config_append = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_config_append = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_dir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_dir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.077926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.078928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.079956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.080041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.080711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.080849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.080956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.081957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] publish_errors = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] publish_errors = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.082992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] standard_threads = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] standard_threads = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.083950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_journal = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_journal = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_json = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_json = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_stderr = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_stderr = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_syslog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.084978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_syslog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.085053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.085129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 10:41:59.085231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.085956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.086991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.087992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.088939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.089873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.090863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.091865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.092964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.093999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.094681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.095815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.096450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.096575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.096654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.096810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.096888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.097958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.098978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.099926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.100869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.101942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.102968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.103941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.104866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.105959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.106976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.107932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.108855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.109932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.external = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.110941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.password = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.token = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.111990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.external = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.112890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.password = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.token = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.113834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.114946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.115929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.116982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.117927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.118888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.119879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.120969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.121997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.122878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.123974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.124990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.125927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.126951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.127941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.128879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.129966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.130987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.131959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.132870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.133975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.134990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.135934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.136877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.137964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.138990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.139950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.140892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.141992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.142069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.142223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.142301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.142458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.142538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.143949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.144964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.145918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.146876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.147900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.148969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.149973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.150944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.151946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.152870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.153976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.154923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.155988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.156924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.157880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.158043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.158121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.158302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.158965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.159882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.160998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.161932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 10:41:59.162392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.162859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.163099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.163333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 10:41:59.163502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 10:41:59.190969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.191098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.191202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.191314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.191429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.191528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.201081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.201192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.230307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.230539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.230940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.231171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.282464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.282577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.333324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.333556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.333916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.334153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.397874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.398947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.399028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.399160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.422955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.423680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.434700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.434880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.447879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.448937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.449021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.449146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.449229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.449358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.449572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 10:41:59.465891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.466104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 10:41:59.466442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.466641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 10:41:59.466739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.466914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.466999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 10:41:59.467174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.467258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.467432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.467518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 10:41:59.467690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.467850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.467983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.468232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.468493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.479746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.480050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.480320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.480550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.480690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.480824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.481083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.481363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.487494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.488324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.490575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.490862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.491145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.491364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.491495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.491634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.491787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.491943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.492879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.493802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.494075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.516547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.518431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.518592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.518817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.519944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.520229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.523445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.523728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.524033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.524915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.525204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.525486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.530667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.530959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.532304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.532633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.535105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.535410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.535678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.535987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.536274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.536571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.536741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.536875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.537016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.537157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.537286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.537535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.537810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.538089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.538359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.538628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.553118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.553395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.554421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.554701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.557436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.558782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.560640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.560938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.561930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.562227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.565869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.567280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.569070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.570478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.572240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.572540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.572835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.573129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.573418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.573672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.573809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.573977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.574273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.574559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.574836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.575109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.589767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.590061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.590328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.590588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.591203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.591493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.591781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.592048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.598233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.599665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.599805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.601723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.602450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.602450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.602873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.603168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.603460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.603751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.604051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.604342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.604629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.604922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.605213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.605490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.616001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.616287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.616567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.616848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.617128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.617389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.617475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.617660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.617742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.617936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.618214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.618492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.618766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.619039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.634328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.635886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.637328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.638829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.640284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.641784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.641879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.643341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.643423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.643634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.643946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.644891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.645176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.645457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.650351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.651852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.651982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.653470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.654908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.655205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.655489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.655787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.656417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.656710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.657001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.657279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.664529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.664910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.665354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.665904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.666196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.666477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.672258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.672544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.672823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.673099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.673309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 10:41:59.673392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.673596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.673680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.673893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.674550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.674707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.674799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.674978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.675948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.676202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.676295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.676480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.676561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.676759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.677021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.677294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.677554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 10:41:59.710074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5580d08d79c0 pid: 81571 (default app) Aug 11 10:41:59.711235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5580d08d79c0 pid: 81570 (default app) Aug 11 10:41:59.713337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8e640-f520-47f0-a51f-0cb21fd51c97 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:41:59.713622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8e640-f520-47f0-a51f-0cb21fd51c97 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:41:59.713895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8e640-f520-47f0-a51f-0cb21fd51c97 None None] PATH_INFO: `/v3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:41:59.718203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Thu Aug 11 10:41:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:08.329488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:08.330556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:08.330556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:08.334931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:08.339939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:08.340340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:08.340519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:08.411003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: INFO dbcounter [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] Registered counter for database keystone Aug 11 10:42:08.431827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG oslo_db.sqlalchemy.engines [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81571) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 10:42:08.437891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writer thread running {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 10:42:08.618424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81571) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 10:42:08.620541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81571) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 10:42:08.672928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:08.674546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:08.804172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 11 10:42:08.948642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:09.016809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: INFO dbcounter [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] Registered counter for database keystone Aug 11 10:42:09.035592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG oslo_db.sqlalchemy.engines [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 10:42:09.041712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writer thread running {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 10:42:09.156889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7880d-ee0e-45de-93e3-253e44196d41 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:09.161609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:09.162217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:09.162493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:09.162763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:09.170560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:09.221811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 10:42:09.222095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf5a82-f0a7-4afb-b08b-07edaf131f0c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:09.231200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Aug 11 10:42:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:42:09.237668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:09.251715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b909a4-51c3-4b58-80da-3d0c85d7aad8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:09.256089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:09.257197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:09.257475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:09.257745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] PATH_INFO: `/v3/projects/345ef69aeca14be9ac484aeebec744ed` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:09.259159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] RBAC: Authorizing `identity:get_project(project_id=345ef69aeca14be9ac484aeebec744ed)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:09.309386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 10:42:09.309725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d15a1-1a92-48e9-9373-ad6647aace4c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:09.314797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Thu Aug 11 10:42:09 2022] GET /identity/v3/projects/345ef69aeca14be9ac484aeebec744ed => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:11.273470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368112-57dc-4df9-bf81-f16493d1fe5f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:11.274013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368112-57dc-4df9-bf81-f16493d1fe5f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:11.274013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54368112-57dc-4df9-bf81-f16493d1fe5f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:11.278122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:11.283444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:11.283848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:11.284016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:11.305184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:11.306670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:11.314962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 11 10:42:11.335831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368112-57dc-4df9-bf81-f16493d1fe5f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:11.348428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54368112-57dc-4df9-bf81-f16493d1fe5f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:11.352545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:11.353125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:11.353399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:11.353665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:11.361653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:11.363382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11338-b948-4d22-b6f3-cb823d74df94 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:11.371900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Thu Aug 11 10:42:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:42:11.377645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:11.386330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21923f5b-5264-41c0-b839-fae23dae1f34 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:11.390618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:11.391226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:11.391505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:11.391798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:11.393136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:11.394919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39d3e1-40dd-4196-919b-d4b9d8f98341 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:11.396442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Thu Aug 11 10:42:11 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:13.416902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a208-005d-4806-9ba7-a8c07415a6dd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.417449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a208-005d-4806-9ba7-a8c07415a6dd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.417449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7a208-005d-4806-9ba7-a8c07415a6dd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.418774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:13.424478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.424874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.425027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.443089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:13.444280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:13.473521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599f6d-077c-4c20-bc75-e693bde246aa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.473807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599f6d-077c-4c20-bc75-e693bde246aa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.474084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9599f6d-077c-4c20-bc75-e693bde246aa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.475379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:13.480439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.480716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.481154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.522870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:13.523426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 10:42:13.523888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 10:42:13.544767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:13.568253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:13.570353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:13.571969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:13.648125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5eeb9-afd0-4b31-a8a1-e0822492addc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:13.669410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:13.682040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:13.682663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.682940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.683212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.691278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:13.694554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:13.702842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:13.704007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:13.705121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:13.709536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:13.789003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:13.793509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bbecb-9916-4d72-be09-7dfc8aa383a8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:13.795773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:13.807857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5c55f-dc78-4895-84a2-7eb64c6c7944 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:13.830143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:13.830990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.831276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.831547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.841660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:13.846638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:13.857472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:13.862752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:13.896871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:13.898374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:13.960385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c2c72-b12c-4ca7-b325-8dd11ac2c4fe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:13.995086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 10:42:14.000868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:14.004776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:14.100001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:14.100626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:14.100904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:14.101173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:14.102291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:14.103714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:14.104309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6acb5-2b1e-4410-9a88-42108f39991c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:14.105284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:13 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:14.111218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:14.125681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:14.138068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:14.138911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:14.143855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:14.143855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:14.157259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:14.158729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:42:14.176851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:14.177795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:42:14.178637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:42:14.178637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:42:14.178637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:14.178637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:14.183684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:14.209180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44acf11-4bd4-4901-96f1-935e2e9ac571 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:14.238122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:14.243723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:14.244365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:14.244645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:14.244919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:14.246028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:14.247614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:14.249178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:14.254948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:14.281608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7265a7-3826-4362-b1ab-6b1a6bfa1c49 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:14.313958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:14.314561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:14.314839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:14.315112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:14.315748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:14.316080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:14.317441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c408e7-4b79-4602-8bb3-3425fca79f8a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:14.334511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:14.334629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:14.355388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939fbc-50fd-4632-9c99-b414fdbc54ff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:14.397280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:14.397896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:14.398172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:14.398438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:14.399166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.domain.id=default, user.id=697f0ea802614a91b28dfb28d2104fff) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:14.399468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:14.400974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d7c96e-041f-4fd8-8951-649973af9eb1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:14.408911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Aug 11 10:42:14 2022] GET /identity/v3/role_assignments?user.id=697f0ea802614a91b28dfb28d2104fff&scope.domain.id=default&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:16.628984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd46db-1133-4520-a57c-2f80db17c8d8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:16.629485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd46db-1133-4520-a57c-2f80db17c8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:16.629592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd46db-1133-4520-a57c-2f80db17c8d8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:16.630886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:16.636213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9116-79d6-4aca-8e5c-e7e8f8e19a09 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:16.636646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9116-79d6-4aca-8e5c-e7e8f8e19a09 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:16.636761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d9116-79d6-4aca-8e5c-e7e8f8e19a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:16.664812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed1d9116-79d6-4aca-8e5c-e7e8f8e19a09 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:16.666276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed1d9116-79d6-4aca-8e5c-e7e8f8e19a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:16.750210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:16.788581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd46db-1133-4520-a57c-2f80db17c8d8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:16.905240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc6c67-26ed-4b1a-b420-f93a2e9e0410 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:16.906134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc6c67-26ed-4b1a-b420-f93a2e9e0410 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:16.906890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc6c67-26ed-4b1a-b420-f93a2e9e0410 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:16.913668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 13/24] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:16.920191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:16.920760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:16.921093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:16.930815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd46db-1133-4520-a57c-2f80db17c8d8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:16.950141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:16.956796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:16.958021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:17.004924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.005872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.006298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.006724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.033397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:17.040159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:17.041060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:17.041822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:17.049100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 12/25] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:17.055175 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.114560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 14/26] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:17.192938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.202231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:17.229136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61adb20-6243-4442-b32e-921ca465eba9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:17.364730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da1f8a-52be-4eb0-99c4-77220114e2d2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:17.406848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.407647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.408041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.408552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.409275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:17.410839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3f1be-f680-43b7-afa0-59ae7f61e483 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:17.414864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.415641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.416084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.416510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.417316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:17.417769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:17.420216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf041f-2b17-4289-99d3-aa27e704b3b7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:17.433422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 13/27] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:17.434850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58a4d0-46fe-454c-b28e-7c83d77c5380 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.435281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58a4d0-46fe-454c-b28e-7c83d77c5380 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.435695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58a4d0-46fe-454c-b28e-7c83d77c5380 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.437024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 14/28] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:17.438149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050321-0e80-47a1-8bbc-45d601aae82f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.439139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050321-0e80-47a1-8bbc-45d601aae82f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.439592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050321-0e80-47a1-8bbc-45d601aae82f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.440962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 15/29] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:17.441818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050321-0e80-47a1-8bbc-45d601aae82f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.489454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:17.564660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050321-0e80-47a1-8bbc-45d601aae82f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:17.615806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.616856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.617314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.617758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.619519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:17.621391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef73696-7e43-4a85-8bbc-8c48f74b6b80 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:17.623258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 16/30] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:17 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:17.624651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.625138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.625616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.647420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:17.649758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:17.667610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 15/31] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Aug 11 10:42:17 2022] POST /identity/v3/roles => generated 219 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:42:17.669136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.669430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.669707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.696181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:17.697877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:17.737320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 17/32] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:17.738560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.770594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:17.779415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 16/33] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:17.781927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.790433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:17.861448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665158a7-1507-45cf-972c-9e2df6c2a493 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:17.900211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.901279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.901845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.902301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.903857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:17.905498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:17.907565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 18/34] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:17.908574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:17.927866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08223a9b-5c29-4b62-a334-e4279445826d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:17.967327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:17.968000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:17.968285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:17.968561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:17.980096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:17.981866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:42:18.000284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:18.001798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:42:18.002598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:42:18.002598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:42:18.002598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:18.002598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 17/35] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:18.002598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.030114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c35e85-2951-4632-9842-b2de0c944f64 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.033183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:18.054351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:18.069970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.070841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.071296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.071730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.072635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.074169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2323211-42d9-4940-8c15-f03f14ad99c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.126980 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b9594-ed47-4aba-8f0d-db37ae3f6cab None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.184366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.185038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.185409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.185721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] PATH_INFO: `/v3/domains/default/users/697f0ea802614a91b28dfb28d2104fff/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.187869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=697f0ea802614a91b28dfb28d2104fff, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.189498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.269800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 18/36] 10.222.0.48 () {58 vars in 1361 bytes} [Thu Aug 11 10:42:18 2022] PUT /identity/v3/domains/default/users/697f0ea802614a91b28dfb28d2104fff/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:18.270782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.281414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 19/37] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Aug 11 10:42:17 2022] POST /identity/v3/roles => generated 225 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:18.283428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65997-296f-4c75-86d5-4f8b755cc0cd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.283722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65997-296f-4c75-86d5-4f8b755cc0cd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.284051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65997-296f-4c75-86d5-4f8b755cc0cd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.285429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 20/38] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:18.300652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2cc15-1eb4-4945-80eb-684fe6e322e9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.301056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2cc15-1eb4-4945-80eb-684fe6e322e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.301352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2cc15-1eb4-4945-80eb-684fe6e322e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.340467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-64b2cc15-1eb4-4945-80eb-684fe6e322e9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:18.343852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64b2cc15-1eb4-4945-80eb-684fe6e322e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:18.482181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 21/39] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:18.486572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8cda0-a73c-4971-8630-b8526339390a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.489319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e98dd-a861-425c-9dab-86d50205124e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.489619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e98dd-a861-425c-9dab-86d50205124e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.489901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e98dd-a861-425c-9dab-86d50205124e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.491287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 22/40] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:18.494787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:18.498114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.498514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.498898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.520631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:18.521838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:18.525725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.526346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.526624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.526894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.527542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:18.527949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.529387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.550203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 19/41] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:18.550988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.609870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 23/42] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:18.611499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.647070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:18.669264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854995-f99f-4573-8ff9-f349a9f47d43 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.705017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.705928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.706345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.706760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.708065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.709730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.716748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 20/43] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:18.717690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.725828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebb3db-590a-40e4-9fa4-296f2171c229 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.756124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:18.764080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.764659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.764946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.765265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.772681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.772681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.823252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43026fc-2ea1-47c0-876e-317cf56ab866 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:18.857419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:18.858588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:18.859320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:18.859826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:18.860700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:18.862308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0eabab-3b5d-43fb-b920-b2d54f3624c9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:18.863670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 24/44] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:42:18 2022] POST /identity/v3/projects => generated 301 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:18.866317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:18.950433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 21/45] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Aug 11 10:42:18 2022] POST /identity/v3/roles => generated 223 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:18.972739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:19.002264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2edd-2998-456d-a0df-403a5d8ba880 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:19.042895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:19.043595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:19.043923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:19.044256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:19.044930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:19.046556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc180f-e77e-446a-af7f-bd036e5aa8de None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:19.136048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 25/46] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:42:18 2022] POST /identity/v3/projects => generated 312 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:20.342383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890050d-fe58-4db5-b1e5-e2ed2a24e3af None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:20.342905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890050d-fe58-4db5-b1e5-e2ed2a24e3af None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:20.342987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890050d-fe58-4db5-b1e5-e2ed2a24e3af None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:20.344314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 22/47] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:20.349875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:20.350274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:20.350557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:20.368077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:20.369263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:20.433228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 26/48] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:20.455902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890050d-fe58-4db5-b1e5-e2ed2a24e3af None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:20.550940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:20.558330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890050d-fe58-4db5-b1e5-e2ed2a24e3af None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:20.591518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:20.592175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:20.592508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:20.592921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:20.605542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:20.610302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:20.611159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:20.611959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:20.613168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 23/49] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:20.617763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:20.673902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:20.714912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616a037-899a-40bb-84c3-a4135d21c8b1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:20.748555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:20.749162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:20.749441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:20.749715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:20.750376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:20.751258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:20.752670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:20.760584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 27/50] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:20.766028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:20.863752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcda6ab-5194-4b18-b463-94b53670e57f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:20.897642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:20.898267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:20.898544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:20.898815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:20.899964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:20.901410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:20.903125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 24/51] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:20 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:20.908405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:21.004497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa7420-5d42-469b-b614-6e30efb19ac9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:21.023926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:21.038160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:21.041613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.041613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.041613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.041613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:21.041954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba314ca5-fad8-412c-9392-79883dbe2cc0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:21.043401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 28/52] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:21.049161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:21.072647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:21.140424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24ff73-d58f-4976-be7c-c7df7af7630b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.140655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24ff73-d58f-4976-be7c-c7df7af7630b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.140894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24ff73-d58f-4976-be7c-c7df7af7630b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.142794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 29/53] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:21.147455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.147735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.148039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.148127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c500a-f2e5-4bb0-ba96-4c50765b531d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:21.165347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:21.166529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:21.182200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:21.182805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.183088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.183361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.184112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.domain.id=default, user.id=697f0ea802614a91b28dfb28d2104fff) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:21.184422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:21.185843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:21.191963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 25/54] 10.222.0.48 () {56 vars in 1322 bytes} [Thu Aug 11 10:42:21 2022] GET /identity/v3/role_assignments?user.id=697f0ea802614a91b28dfb28d2104fff&scope.domain.id=default&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:21.235912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 30/55] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:21.258717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:21.347499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:21.362026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452fee-2d3c-4fba-b311-b72ec6e8a4ec None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:21.396762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:21.397319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.397600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.397870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.398959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:21.400428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af9ab3-69bd-45d6-8de4-694838242bf5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:21.401985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 26/56] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:21.408030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:21.503490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a0308-1dfd-4d28-83af-8ca0502100c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:21.519881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:21.535899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:21.536501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:21.536783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:21.537057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:21.537703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:21.539139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629788d-2acc-4c12-af34-ccd33caaea16 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:21.623074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 31/57] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Aug 11 10:42:21 2022] POST /identity/v3/projects => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:42:23.591187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fe85-76a1-4987-bef7-5b2f2942edc0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:23.591737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fe85-76a1-4987-bef7-5b2f2942edc0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:23.591737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67fe85-76a1-4987-bef7-5b2f2942edc0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:23.593110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 27/58] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:23.598607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:23.599029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:23.599163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:23.616629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:23.617813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:23.680991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 32/59] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:23.705269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67fe85-76a1-4987-bef7-5b2f2942edc0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:23.803470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67fe85-76a1-4987-bef7-5b2f2942edc0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:23.835381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:23.836022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:23.836305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:23.836579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:23.837716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:23.839144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9f006-9a39-4aef-9852-8da61694e680 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:23.840713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 28/60] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:23.846355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:23.940014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:23.942761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8655-d5e6-4e08-9717-dc9f32ea38de None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:23.975640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:23.976267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:23.976550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:23.976823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:23.977472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:23.978898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335512e-9ca3-4390-b1f2-546863c5569e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:24.107072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 33/61] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:23 2022] POST /identity/v3/users => generated 275 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:27.355541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc5bd-561c-41c0-a74d-57b20a92d232 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.356609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc5bd-561c-41c0-a74d-57b20a92d232 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.357086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc5bd-561c-41c0-a74d-57b20a92d232 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.358643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 29/62] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:27.364775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b1c8-36e6-4224-aab8-635a519cf04d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.365583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b1c8-36e6-4224-aab8-635a519cf04d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.366182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b1c8-36e6-4224-aab8-635a519cf04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.370217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08038b6-fe3d-450f-9c46-39ad81fa14c1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.371072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08038b6-fe3d-450f-9c46-39ad81fa14c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.372118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08038b6-fe3d-450f-9c46-39ad81fa14c1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.373596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 30/63] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:27.381072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.381641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.382107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.389773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0a36b1c8-36e6-4224-aab8-635a519cf04d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:27.391263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a36b1c8-36e6-4224-aab8-635a519cf04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:27.413188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:27.414407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:27.452436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:27.485486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 34/64] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:27.490299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702ba0-efa4-4b27-8717-66f934376051 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.490587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702ba0-efa4-4b27-8717-66f934376051 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.490863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d702ba0-efa4-4b27-8717-66f934376051 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.492317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 35/65] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:27.511618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 31/66] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:27.512601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.512892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.513167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.520644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:27.549757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:27.551283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:27.626391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 36/67] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:27.627536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:27.647340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:27.663989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f668d-cb2d-4a7d-803b-5122389a5eb1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:27.704865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:27.706294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.706753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.707172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.721011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:27.727667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:27.728733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:27.729695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:27.731875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 32/68] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:27.732802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:27.745464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a8b8b-96f4-4364-b08f-2e04a9085236 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:27.753593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:27.784860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:27.786097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.786578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.787207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.799168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d0056db6-eb7b-46c2-bca5-4026db2d05c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:27.803997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:27.804941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:27.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 37/69] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:27.807524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5171-a39d-4a69-a436-bc99e43a3acf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.807837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5171-a39d-4a69-a436-bc99e43a3acf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.808123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5171-a39d-4a69-a436-bc99e43a3acf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.811803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 38/70] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:27.811803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5171-a39d-4a69-a436-bc99e43a3acf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:27.843742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4e284-7696-4462-9542-6ecbe8843073 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:27.877484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:27.878037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.878314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.878582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.886782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:27.888620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-41f2b93e-30fa-41cc-a050-2d42998a63d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:27.893873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:27.894719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:27.895472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 33/71] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:27.902539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8c216-45d5-4c6c-ab20-64fe0469be18 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.902539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8c216-45d5-4c6c-ab20-64fe0469be18 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.902539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8c216-45d5-4c6c-ab20-64fe0469be18 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.902539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 34/72] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:27.902539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8c216-45d5-4c6c-ab20-64fe0469be18 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:27.910757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5171-a39d-4a69-a436-bc99e43a3acf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:27.944874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:27.945483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.945764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.946038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.946706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:27.947011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:27.948429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65fc30-05d1-42e6-aba6-150977ffbfba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:27.956824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 39/73] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:27.957961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:27.958249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:27.958523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:27.975391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:27.976608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:27.976752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.006658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8c216-45d5-4c6c-ab20-64fe0469be18 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.039854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.040466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.040747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.041021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.041686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:28.041989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.042571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 40/74] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:28.043382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f90f15-54bb-4be6-b5bf-1af5c87e0f08 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.043469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.051934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 35/75] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.057292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.058038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.058329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.077272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:28.079020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:28.107721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.140771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf5d54-a046-4098-811d-1642a816ca96 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.144202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 36/76] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:28.145023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.176499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.177164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.177441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.177712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.178360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:28.178661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.180062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.188319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 41/77] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.189029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.206759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.243465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beeddbf-76e5-40fc-b9be-a8eb6b2bc1fe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.276688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.277303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.277582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.277854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.287065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd02a-3b9f-4c27-850c-0eb024ef064b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.290470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.291932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.293519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 37/78] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.294375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.319405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.325148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.325774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.326054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.326327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.327453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.328905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.330539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 42/79] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.331213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.392750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2ade4-4a4f-4243-8602-27bfa5e836f6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.415003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.426986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.427536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a368e3-3403-491f-b32f-39dac484a237 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.427976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.427976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.428227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.438526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:28.443174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:28.444092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:28.444870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:28.445525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.467034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.467659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.467963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.468234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.469608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.471023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.472745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 43/81] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.473403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.537942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.546962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf6e2-cef6-4d56-b783-b1cceb623059 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.571007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c4ef7-ca43-47c0-83e6-5aa6014f3903 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.580000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.580628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.580906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.581181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.582309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.583789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.585547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 39/82] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.586126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.611307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.611945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.612226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.612501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.638964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.678760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.680188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.686838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e043f-7a99-4e84-ab64-81d13f030aae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.720481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.721082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.721366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.721640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.742996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.776973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 44/83] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:28.778052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.792429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.793887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.839916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.873745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d023-bd2d-4500-b92a-621228d367db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:28.889718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 40/84] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:28.890595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.909585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:28.910180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:28.910456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:28.910722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:28.911369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:28.911671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:28.913059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:28.920660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 45/85] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:28.921429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:28.948802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:28.989692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839cfa2-dc41-47e1-bb50-d474141a50d6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.019985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8bb9-076d-4f82-a824-1264fcdd2905 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.021988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.022587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.022864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.023135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.023789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.025212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.053068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.067734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.067734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.067734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.121355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.122816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.130421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 41/86] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:28 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:29.144874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.161192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.162867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.243940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 46/87] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:29.244728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.272019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22876-8b79-4010-9c2f-73e0f576ee13 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.316545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.316788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.317074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.317351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.319456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.321031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.322694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 42/88] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.323523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.363204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92f427-7787-4992-9d75-a8e9524e367f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.397584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.408413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.409058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.409394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.409677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.410809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.411241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.412365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.414203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 47/89] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.415171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.445392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349f8e-76e3-48d6-b51a-13afab981966 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.485430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.486527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.487012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.487487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.488883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.490539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.492660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 43/90] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.493617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.534285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9cbfb-6724-4e98-b34d-97569eaafbc0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.571593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.572376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.572623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.572900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.574143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.575554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.577473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 48/91] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.582311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.650728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43237-90ee-4005-9a21-8d8dcd297395 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.654578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.694308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.695254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.695605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.695971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.697165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:29.697549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.699596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.703063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93445d9-25d1-4354-b018-92b5c280b638 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.707432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 44/92] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=933643980c98414dad5713dc8820df36 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.712694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.730337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:29.780598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.781273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.781563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.781846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:29.782637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=697f0ea802614a91b28dfb28d2104fff) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:29.782953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:29.784522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:29.794986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 49/93] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/role_assignments?user.id=697f0ea802614a91b28dfb28d2104fff&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:29.800808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:29.853974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7526-6459-4eb6-85c6-70479b7989b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:29.899079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:29.900037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:29.900489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:29.900933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:30.002125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca9ff1-73a0-46a4-9a68-5bea1ed2ed90 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:30.041000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:30.041863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:30.043535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:30.052608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:30.057047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:30.057793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:30.058077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:30.058372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:30.059149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:30.059461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:30.060943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:30.067475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 50/94] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=e0028b79d128480ba32f23616eb613c1 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:30.162110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 45/95] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:30.168381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:30.325181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd4f43-abb5-4fb4-9001-f3084aa91a14 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:30.374624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:30.375843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:30.376134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:30.376410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] PATH_INFO: `/v3/projects/877c42fef9794410a5e18bcdaf477be2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:30.380497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] RBAC: Authorizing `identity:get_project(project_id=877c42fef9794410a5e18bcdaf477be2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:30.381983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a329a0-af24-4b53-98cb-5b4535f7e07a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:30.383663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 51/96] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:30 2022] GET /identity/v3/projects/877c42fef9794410a5e18bcdaf477be2 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:30.389909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:30.541331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82260ee7-3d6d-4d24-a999-b7184e591f4a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:30.554281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:30.591425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:30.591995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:30.592282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:30.592558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:30.593305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=877c42fef9794410a5e18bcdaf477be2, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:30.593640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:30.595400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9303d1-4487-496b-b3ed-30f577bc5efc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:30.604325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 46/97] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:30 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=877c42fef9794410a5e18bcdaf477be2&role.id=933643980c98414dad5713dc8820df36 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:31.911650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a385-a909-429b-8c5d-111377c50573 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:31.912674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a385-a909-429b-8c5d-111377c50573 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:31.913158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a385-a909-429b-8c5d-111377c50573 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:31.914715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 52/98] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:31.919930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:31.920759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:31.921232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:31.942968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:31.944522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:32.029754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 47/99] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:32.066140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5a385-a909-429b-8c5d-111377c50573 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.138974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:32.196432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5a385-a909-429b-8c5d-111377c50573 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:32.237340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:32.237971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.238251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.238528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.253091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9a5b5fe1-444e-4fcb-ae38-6fe3a132650f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:32.259171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:32.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:32.261098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 53/100] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:32.268045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.389076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:32.392347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488276a9-0a59-49c3-a8e5-656d812855c4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:32.397761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9da67-88c9-4f20-8a36-626df45fd2a5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.398050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9da67-88c9-4f20-8a36-626df45fd2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.398323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9da67-88c9-4f20-8a36-626df45fd2a5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.399681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 54/101] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:32.404631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fed73-9dfc-48fe-b602-c48d485a1c9c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.404905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fed73-9dfc-48fe-b602-c48d485a1c9c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.405146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fed73-9dfc-48fe-b602-c48d485a1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.431054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de9fed73-9dfc-48fe-b602-c48d485a1c9c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:32.433073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de9fed73-9dfc-48fe-b602-c48d485a1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:32.469354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:32.469983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.470268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.470541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.471217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:32.472537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:32.473947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:32.487282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 48/102] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:32.495989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.559239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 55/103] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:32.560528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01003d4-98d7-4e7a-aeea-cb01c8a89453 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.560815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01003d4-98d7-4e7a-aeea-cb01c8a89453 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.561090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01003d4-98d7-4e7a-aeea-cb01c8a89453 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.566522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 56/104] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:32.573096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.573379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.573657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.620918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:32.622444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:32.683173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f340-62d1-401c-a930-1c4044699b41 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:32.711816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:32.731105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 57/105] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:32.731954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.772497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:32.773146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.773431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.773710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] PATH_INFO: `/v3/users/6ab36f83a70143998231c2310a7c536a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.803188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab36f83a70143998231c2310a7c536a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:32.805401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:32.807105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 49/106] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/users/6ab36f83a70143998231c2310a7c536a => generated 289 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:32.807923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.830827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:32.917798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d103cf9-56c2-4cea-b947-4d008a9464d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:32.918323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56d7c-3845-452a-9f1d-d4e22d04373f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:32.961384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:32.962073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.962358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.963309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.965250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:32.966709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:32.968314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 50/107] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:32.969387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:32.985316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:32.986316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:32.986806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:32.987281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:32.997821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-749bb22d-cd3d-4bd3-a9ad-04dd9671dccf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:33.003450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:33.004489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:33.005276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 58/108] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:33.006321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613cc01-9a6a-46fa-bfac-f30f9ba49341 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.006605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613cc01-9a6a-46fa-bfac-f30f9ba49341 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.006878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613cc01-9a6a-46fa-bfac-f30f9ba49341 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 59/109] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:33.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613cc01-9a6a-46fa-bfac-f30f9ba49341 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.060196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.069950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.081728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf96bf-a31c-44f1-b356-23f8e6016caf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.116643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.117201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.117558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.117754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.122611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613cc01-9a6a-46fa-bfac-f30f9ba49341 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.157757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.158366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.158652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.158928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.159571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.161090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d82eaa-73b3-49b5-b1b6-ffd844fac06e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.187049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.188577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab8d65-9075-46fc-8dbc-7c4360369425 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.250642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 60/110] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:42:33 2022] POST /identity/v3/projects => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:33.251627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8cf3-8962-49da-b320-6141b1aee70c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.251940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8cf3-8962-49da-b320-6141b1aee70c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.252216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d8cf3-8962-49da-b320-6141b1aee70c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.253418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 61/111] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:33.258232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d8cf3-8962-49da-b320-6141b1aee70c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.296703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.312074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 51/112] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:33.314009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212c559-6d1f-41eb-8254-69f61d9398e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.314561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212c559-6d1f-41eb-8254-69f61d9398e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.315087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212c559-6d1f-41eb-8254-69f61d9398e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.337645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a212c559-6d1f-41eb-8254-69f61d9398e5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:33.339181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a212c559-6d1f-41eb-8254-69f61d9398e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:33.388040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.402542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d8cf3-8962-49da-b320-6141b1aee70c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.420164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 52/113] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:33.422452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.422908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.423378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.446291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:33.447892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:33.448213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.448831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.449104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.449381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.450057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:33.450368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.454147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.465192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 62/114] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:33.466041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.548674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 53/115] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:33.549304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.610138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7637a632-95c5-439d-8221-1ce9017c8f82 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.625096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.652412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.653054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.653337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.653610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.654757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.656797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.658970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 63/116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:33.659777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.685315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0e017-e4a8-4e39-aac0-849c058c36cf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.690609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.725774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.726558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.726844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.727117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.737925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:33.742704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:33.743551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 54/117] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:33.745333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.782992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17de3-9438-4f19-8f9d-4251e19b3f5b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.817964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.818518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.818800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.819075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.820232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.821661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.823404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 64/118] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:33.824078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.861701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f95da-ff7a-446d-854c-4a79222cf3be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.870973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.896871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.897518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.897807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.898080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.909883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fb4787b2-09cc-4a51-b14d-60b7bc580fd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:33.915061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:33.916915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 55/119] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:33.918495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269df4d-ccaa-45f0-83cb-69d0bc0230bd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.920825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269df4d-ccaa-45f0-83cb-69d0bc0230bd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.920825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269df4d-ccaa-45f0-83cb-69d0bc0230bd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.920825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 56/120] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:33.920825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269df4d-ccaa-45f0-83cb-69d0bc0230bd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:33.931007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:33.941171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d46d46-075f-4617-8e58-c5efd976ab6f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:33.978718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:33.979392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:33.979674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:33.979979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:33.980712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=6ab36f83a70143998231c2310a7c536a) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:33.981023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:33.982461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:33.988048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 65/121] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/role_assignments?user.id=6ab36f83a70143998231c2310a7c536a&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:33.988847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.068113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a269df4d-ccaa-45f0-83cb-69d0bc0230bd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.105443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511105-e1c2-4958-9abf-1e643e265e8d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.123072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.124296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.125304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.125749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.127225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:34.127722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.129406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22377-b044-4a4e-9cf9-bb7f835973bb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.143158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 57/122] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.145402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.147616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.147616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.148852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.149665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.150055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.150449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.151701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.154357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.158795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 66/123] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:33 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.159697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.169997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:34.171254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:34.194448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.209391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.263202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 58/124] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:34.264438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.313757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b8b1d-74c3-4d38-88b5-edd2f8002505 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.353969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.355243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.355667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.356498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.357603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:34.358046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.360853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.371858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 67/125] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.372810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.387576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179a189-5fbf-4d33-877d-a7889890bf5a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.431330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.432434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.432821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.433197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.434685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.436953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.439057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 59/126] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.440190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.481003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.488122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.500986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34c503-d03e-4709-9b55-d520ce1063be None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.548724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.549625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.550056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.550476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/ee0c0304aa16452fb5a1106b5a439277/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.553413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user_id=ee0c0304aa16452fb5a1106b5a439277, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.555284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.566124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd885a9-fead-48d7-a7ad-cfb3b325bc84 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.621600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 68/127] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:34 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/ee0c0304aa16452fb5a1106b5a439277/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:34.628859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.635891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.636567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.636897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.637184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.651546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:34.657952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:34.661965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:34.663827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 60/128] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:34.665359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.780465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.796679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a268c3-738f-4cc3-b1a3-11b27337b434 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.839314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.840048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.840335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.840881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.842047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.844408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.846868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 61/129] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.847498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.886784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c651f-eb7a-4b26-b213-014fc5507dfd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:34.925233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:34.925972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:34.926261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:34.926541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:34.927749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:34.929233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:34.931829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 69/130] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:34.932614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:34.953230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:34.984492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3aa3b1-c3a1-416c-be45-e27d543ce851 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.030714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.031912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.032199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.032474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.033405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:35.033722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.035675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.045710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 62/131] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:35.046582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.073877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:35.093770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81dfe8-d51c-4661-bcb8-0cc610be4b25 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.223693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.223693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.223693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.223693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/ee0c0304aa16452fb5a1106b5a439277/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.224186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a931d-d0ba-487b-a542-b881349b7bff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.224280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user_id=ee0c0304aa16452fb5a1106b5a439277, role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.224280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.224431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.224745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.225032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.225312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.226440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.227938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.229864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 63/132] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:35 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:35.230636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.324809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:35.341393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eefed-e721-4f58-bae6-43de38c92318 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.354902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 70/133] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:34 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/ee0c0304aa16452fb5a1106b5a439277/roles/e0028b79d128480ba32f23616eb613c1 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:35.355730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.394355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.395005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.395288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.395570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.398629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.400140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.402178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 64/134] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:35 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:35.408399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.447417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:35.486647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51cccd-8e16-4911-8074-846833f484b8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.528910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.530007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.530470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.531522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/697f0ea802614a91b28dfb28d2104fff/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.534550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user_id=697f0ea802614a91b28dfb28d2104fff, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.536050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.611389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af2db7-f9fa-4222-a7d4-dd8dec20f112 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.646902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.647585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.647902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.648247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] PATH_INFO: `/v3/projects/877c42fef9794410a5e18bcdaf477be2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.649394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] RBAC: Authorizing `identity:get_project(project_id=877c42fef9794410a5e18bcdaf477be2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:35.650981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ca60a-759d-41ba-b579-aeaac7612547 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:35.652821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 65/135] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:35 2022] GET /identity/v3/projects/877c42fef9794410a5e18bcdaf477be2 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:35.653812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d58a-8f34-4d98-9395-2ecd0f9e44e2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.654102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d58a-8f34-4d98-9395-2ecd0f9e44e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.654384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d58a-8f34-4d98-9395-2ecd0f9e44e2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.656297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 66/136] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:35.661490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.661828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.662109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:35.675022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 71/137] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:35 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/697f0ea802614a91b28dfb28d2104fff/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:35.676065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.696934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:35.698167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:35.719379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:35.835722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 67/138] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:35.880915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:35.939328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c3cd4-cfe7-4dd7-a371-84295a596f12 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:35.980601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:35.996578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:35.997610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:35.998154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:35.998614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] PATH_INFO: `/v3/projects/877c42fef9794410a5e18bcdaf477be2/users/ee0c0304aa16452fb5a1106b5a439277/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.005549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=877c42fef9794410a5e18bcdaf477be2, user_id=ee0c0304aa16452fb5a1106b5a439277, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:36.007010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207b9bd-5e2e-4823-844d-d206787b1fe3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:36.014145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d1aa0-f0ec-465b-a4fa-e04a0f6b70d0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:36.060395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:36.061030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.061306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.061579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.062759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:36.064244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:36.066117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 68/139] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:36.071288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:36.098194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:36.138802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 72/140] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:35 2022] PUT /identity/v3/projects/877c42fef9794410a5e18bcdaf477be2/users/ee0c0304aa16452fb5a1106b5a439277/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:36.204223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c62069-6e14-40c2-aceb-1e5cb84ca40f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:36.241245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:36.241882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.242161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.242435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.243152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:36.244703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:36.289492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4243e-0de6-4d9d-9c23-c13766091552 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.289848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4243e-0de6-4d9d-9c23-c13766091552 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.290130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4243e-0de6-4d9d-9c23-c13766091552 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.291802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 73/141] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:36.297018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.297318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.297573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.317656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:36.321130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:36.377248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 69/142] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:36 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:36.417004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 74/143] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:36.440175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:36.525818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:36.566965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535d36-f81c-48e4-9d0e-005edc4e909d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:36.608741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:36.609417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.609694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.609964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.626302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:36.631654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:36.632583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:36.633384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 70/144] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:36.640608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:36.775437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c1240-5337-45cb-8181-58e33b694ff4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:36.799702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:36.810764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:36.816632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:36.823673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 75/145] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:36.829969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:36.934174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcfa82-ae01-454a-9139-3ccc038b3b81 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:36.978641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:36.979282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:36.979560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:36.979884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] PATH_INFO: `/v3/users/6ab36f83a70143998231c2310a7c536a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:36.981036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab36f83a70143998231c2310a7c536a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:36.982470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:36.984129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 71/146] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:36 2022] GET /identity/v3/users/6ab36f83a70143998231c2310a7c536a => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:36.989927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:37.114762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadf23-6e02-4e34-88aa-a25747af3873 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:37.154859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:37.155500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:37.156335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:37.156613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:37.160800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:37.163183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fbe68-0022-4018-9064-ab7ce89bc927 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:37.165954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 76/147] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:36 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:37.172217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:37.179482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:37.309778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345ae-46a1-4def-ab83-305c83345819 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:37.348118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:37.348784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:37.349058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:37.349329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/6ab36f83a70143998231c2310a7c536a/roles/41f5ad14f8214ea6b35c94a515cb80fa` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:37.352006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user_id=6ab36f83a70143998231c2310a7c536a, role_id=41f5ad14f8214ea6b35c94a515cb80fa)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:37.353436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a943b-3bf3-4f3e-858e-f7beda24561d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:37.422631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 72/148] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:37 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/users/6ab36f83a70143998231c2310a7c536a/roles/41f5ad14f8214ea6b35c94a515cb80fa => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:37.548041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af8f1-aa8e-4933-91f9-fcb4518764ac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:37.548430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af8f1-aa8e-4933-91f9-fcb4518764ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:37.548617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3af8f1-aa8e-4933-91f9-fcb4518764ac None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:37.550387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 77/149] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:37.557417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:37.557714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:37.558032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:37.589079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:37.590368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:37.668863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:37.679291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 73/150] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:37.735742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3af8f1-aa8e-4933-91f9-fcb4518764ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:37.813177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:37.877207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3af8f1-aa8e-4933-91f9-fcb4518764ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:37.918358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:37.919048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:37.919330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:37.919601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:37.934211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:37.939523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:37.940444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:37.941655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 78/151] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:37.946689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:38.077030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed895ae-0383-49f8-b708-315132a1e91c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:38.120140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:38.121125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.121614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.122086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.122958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:38.123456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:38.125105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:38.134505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 74/152] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:38.140935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:38.271264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfc078-8cc7-42f2-8a8e-c2a77adacfbf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:38.285923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:38.307168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:38.307819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.308099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.308368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.309509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:38.310959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:38.312675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 79/153] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:38 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:38.318351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:38.400270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:38.458528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cfb6c-d85b-4420-8999-f69a16489deb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:38.499883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:38.500515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.500793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.501062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.583023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:38.584523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb36f1e-ba8a-410b-9570-372aac237d88 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:38.652089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:38.712505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 75/154] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:38.719512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:38.794300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ae0d1-b5aa-45a8-a93f-2be96a5f879b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.796088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ae0d1-b5aa-45a8-a93f-2be96a5f879b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.797771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ae0d1-b5aa-45a8-a93f-2be96a5f879b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.799998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 76/155] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:38.805502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.806104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.806566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.838797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:38.841604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:38.879440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a86d4-adc8-40fb-b3db-e07ad32ebfe2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:38.952288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 77/156] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:38.977361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:38.978002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:38.978282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:38.978554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:38.984941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:38.986426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:38.988309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 80/157] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:38 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:38.989363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:38.993865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.121199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.129654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.141987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81a49b-3cb5-4797-8bf5-12ab5e14bf3b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.210305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.210868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.211143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.211411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.214564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b4f02-2a50-4511-83bd-b761937419ba None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.238652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-aad0fbea-72f8-446f-a80e-a7d0d1ffdd31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:39.245663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:39.246517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:39.247318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 78/158] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:39.248106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0340-b706-455b-ac8c-11829ab06d3c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.249012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0340-b706-455b-ac8c-11829ab06d3c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.249012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0340-b706-455b-ac8c-11829ab06d3c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.250468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 79/159] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:39.257026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.257306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.257580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.281138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.281138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.281437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.281655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.282364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:39.282667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:39.284122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:39.286234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:39.287717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:39.290948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 81/160] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:38 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=933643980c98414dad5713dc8820df36 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:39.291359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.396874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 80/161] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:39.438897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.460564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af99d7-5006-437e-9b66-c8d3d4c8012e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.474596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.510106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.510734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.511014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.511285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.512039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:39.516271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:39.517771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:39.526415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 82/162] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:39.548495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.557800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.584014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f64e66-c96e-4a56-b6d2-9db1917487d5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.629938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.630588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.630865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.631133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.663166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:39.668205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:39.669168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:39.670026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 81/163] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:39.682336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.734882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71760e-3c89-420e-83d6-b03e5ee8143a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.803376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.806397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.806881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.807351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.808732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:39.809444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e49d45-fa3c-4c30-8bb6-d44981bcaf69 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:39.810587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44c0f3-85aa-4264-95c5-54d9685e0300 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:39.812748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 83/164] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:39.814107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e6b52-62da-40bd-b58e-ed80ffb83bd9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.814720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e6b52-62da-40bd-b58e-ed80ffb83bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.815255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e6b52-62da-40bd-b58e-ed80ffb83bd9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.817015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 84/165] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:39.821903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.822435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.822899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.849410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:39.850912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:39.859959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:39.860586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:39.860865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:39.861139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:39.861802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:39.862106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:39.863615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:39.866229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.877846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 82/166] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:39.883862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:39.911134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:39.959200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 85/167] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:39.960095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:40.031140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcc093-34ef-4c22-897f-8f0c3485dc03 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.088490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.089105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.089385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.089661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.096468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1998-d643-46a7-8218-7072ddb83841 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.153350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.153997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.154275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.154546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] PATH_INFO: `/v3/users/697f0ea802614a91b28dfb28d2104fff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.160452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] RBAC: Authorizing `identity:get_user(user_id=697f0ea802614a91b28dfb28d2104fff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:40.161927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:40.163548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 86/168] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/users/697f0ea802614a91b28dfb28d2104fff => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:40.164344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:40.177605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:40.179084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:40.187572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:40.235427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:40.345718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06a52-038d-4398-96b3-be05eb1788b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.377346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 83/169] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:40.378229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:40.435750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.436831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.437300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.437803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.460347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:40.465532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:40.466424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:40.467201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:40.470552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 87/170] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:40.471563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:40.629889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6419b-bf15-4fc8-bb3d-9cd2e5435375 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.631556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:40.640411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:40.644469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0816508-6b6b-4b50-9f4b-4e7b974b7732 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.705139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.705754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.706030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.706299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.716467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.717112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.717392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.717661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.719607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:40.721084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:40.722728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 88/171] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:40 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:40.723487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:40.823476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:40.825035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:40.942824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c93c4-4c4e-4e84-ba1f-200ca59a2136 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:40.976070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:40.979710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:40.980341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:40.980631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:40.980889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:40.981551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:40.981853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:40.983216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12addf99-7938-4d48-a5be-ebe9289450f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:40.987699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 84/172] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:40.988414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.007591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 89/173] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:41.010749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11224-40b6-45bc-a99b-f2bb1def85b4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.010749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11224-40b6-45bc-a99b-f2bb1def85b4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.010749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11224-40b6-45bc-a99b-f2bb1def85b4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.010749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 90/174] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:41.011800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11224-40b6-45bc-a99b-f2bb1def85b4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.075740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.166170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc005d-3aed-4bfb-88f9-b1e3cdc46e9d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.185757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11224-40b6-45bc-a99b-f2bb1def85b4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.207432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.208390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.208861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.209352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.210266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:41.210765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:41.212481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:41.221072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 85/175] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:40 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=e0028b79d128480ba32f23616eb613c1 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:41.222127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.247052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.247737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.248321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.248321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.250299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:41.251784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e26f1-eaf6-439b-8e0b-711842861036 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:41.253549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 91/176] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:41.259477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b0b7-0a27-4ca3-bf5b-426c61ac30ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.259862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b0b7-0a27-4ca3-bf5b-426c61ac30ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.260142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055b0b7-0a27-4ca3-bf5b-426c61ac30ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.301633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b055b0b7-0a27-4ca3-bf5b-426c61ac30ba None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:41.302866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b055b0b7-0a27-4ca3-bf5b-426c61ac30ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:41.363189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.411648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99377-26c4-4f8d-b2c7-6bbdcc1b3594 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.424852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 92/177] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:41.427051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9ecb-2cc0-4d82-a4b2-89fa80e4bd10 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.427326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9ecb-2cc0-4d82-a4b2-89fa80e4bd10 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.427598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9ecb-2cc0-4d82-a4b2-89fa80e4bd10 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.428966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 93/178] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:41.429780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b9ecb-2cc0-4d82-a4b2-89fa80e4bd10 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.455222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.456205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.456652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.457098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] PATH_INFO: `/v3/users/ee0c0304aa16452fb5a1106b5a439277` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.458973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0c0304aa16452fb5a1106b5a439277)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:41.460409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb8b4-34cc-49a2-bc90-e390c1baeeb8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:41.462037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 86/179] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/users/ee0c0304aa16452fb5a1106b5a439277 => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:41.463237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b820-9096-4458-89d0-9149aa19d531 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.463533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b820-9096-4458-89d0-9149aa19d531 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.463906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b820-9096-4458-89d0-9149aa19d531 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.464007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.465224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 87/180] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:41.466311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.466587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.466856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.490190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:41.491378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:41.563409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b9ecb-2cc0-4d82-a4b2-89fa80e4bd10 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.578115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 88/181] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:41.578800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.614946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.615570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.615880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.616146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.616869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=697f0ea802614a91b28dfb28d2104fff) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:41.617171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:41.618587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217a2a3-b1ef-4e0e-ae84-c08739efdd68 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:41.624584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 94/182] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/role_assignments?user.id=697f0ea802614a91b28dfb28d2104fff&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:41.625591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.625871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.626142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.653159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:41.653159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:41.671403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.714952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236ae00-7aa8-4afb-8374-cf84f2807550 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.739431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 95/183] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:41.744765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.754536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.755203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.755487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.755752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.769407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:41.773151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:41.774367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:41.775479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 89/184] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:41.776519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.872383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8c76f-95f3-4ad2-a0a5-564f57d1e821 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.900736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ff020-b7c7-4383-803d-683d9c977cf8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:41.908172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.908776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.909046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.909309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.920809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:41.934220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:41.934839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.935115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.935384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.945180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e0c20b5e-b7ee-4188-9597-5f97fd2ce5b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:41.949861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:41.950706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:41.951520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 90/185] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:41.953710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290284-4681-47eb-ae4c-a1a41370dadc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:41.954244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290284-4681-47eb-ae4c-a1a41370dadc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:41.954730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290284-4681-47eb-ae4c-a1a41370dadc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:41.959872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 91/186] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:41.959872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290284-4681-47eb-ae4c-a1a41370dadc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:41.982721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:41.984153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.006740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.057933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290284-4681-47eb-ae4c-a1a41370dadc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.080065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 96/187] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:42.080877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.090913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.091529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.091835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.092103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.093685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.095355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.097124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 92/188] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.097680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.145031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.185457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa566b-44b7-4301-87d1-946fd5239374 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.194849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c774bca-03c8-4286-bd41-8bd31b53641d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.207381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.219385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.220111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.220390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.220655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.221298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:42.221594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.222940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ec9b5-7de8-407d-b0f0-2c074f25e409 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.229447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.230033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.230305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.230573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.231218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:42.231302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 97/189] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.232121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.232528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.232879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.233087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.234073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.242649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 93/190] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.244628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.251270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:42.252545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:42.316983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 98/191] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:42.317768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.359458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df1c61-3813-4561-b26d-6a129a0f025a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.369284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.392093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.392706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.392978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.393249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] PATH_INFO: `/v3/projects/877c42fef9794410a5e18bcdaf477be2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.394400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] RBAC: Authorizing `identity:get_project(project_id=877c42fef9794410a5e18bcdaf477be2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.395967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.399921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 94/192] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/projects/877c42fef9794410a5e18bcdaf477be2 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.400735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.414938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ce94c-c8be-4bdf-bc63-09188f04319d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.428451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.447828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.448576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.448881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.449143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.449770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.451185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.497331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4d078-4678-40b0-84f9-b50c926c6a79 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.531892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.532507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.532784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.533132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.546446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.547995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.549678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 95/193] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.559820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.559968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 99/194] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:42 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:42.559968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.594604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.671387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bde9-0b67-4ae7-a874-7044359733f9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.671738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caa4d1-4641-4cbc-8585-ea1ad0564aed None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.709899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.710696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.710885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.711324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.711324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.711528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.711837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.712120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] PATH_INFO: `/v3/users/6ab36f83a70143998231c2310a7c536a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.713251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] RBAC: Authorizing `identity:get_user(user_id=6ab36f83a70143998231c2310a7c536a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.714714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.716327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 96/195] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/users/6ab36f83a70143998231c2310a7c536a => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.717095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.723223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:42.728559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:42.729485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:42.730369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 100/196] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:42.731196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.765446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.841381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ccadd5-4a1d-4f70-ba75-4fedb1ccf84a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.847166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c463-f8d4-4413-a23c-553891bd27cf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:42.851314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:42.886501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.887114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.887394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.887668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.891751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:42.892405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:42.892684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:42.892952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:42.893683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=877c42fef9794410a5e18bcdaf477be2, user.id=ee0c0304aa16452fb5a1106b5a439277) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:42.893982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.895422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:42.901769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 97/197] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/role_assignments?user.id=ee0c0304aa16452fb5a1106b5a439277&scope.project.id=877c42fef9794410a5e18bcdaf477be2&role.id=933643980c98414dad5713dc8820df36 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:42.902693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:42.984439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:42.989564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.027112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4738e-edd1-4f78-9344-b5a3dcb79a98 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.028587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.065073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.065771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.066056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.066512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.104395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.110058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 101/198] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:43.111190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.152592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.155795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.244339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5428a6-35b5-4a2e-8140-bb02cf8dc139 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.294071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.303477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 98/199] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:43.307665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.307812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.307812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.307812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.307812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.308071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:43.308071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.308472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.323523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 102/200] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:43.324799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.412631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.455667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014004b-0341-43be-b520-cb4a0b9ade3d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.464491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eff6e-5740-457f-af72-77e624ac3d09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.501395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.502333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.502701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.503094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.505157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.506630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.508554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 99/201] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:43.513360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.513507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.513507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.513507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.513731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.516349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.517738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.519483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 103/202] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:43.523902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.680297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.726341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf8ea7-4f6f-49a1-9a7e-37a7b7773316 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.726948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee4f64-2c88-4605-87c7-bc4818d977fa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.764972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.765791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.766138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.766512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.767368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=9f013eb15a6e4ce5abef9fe0396378b8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:43.767798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.769477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.772326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.773761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.774315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.774823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.775890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 104/203] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/role_assignments?user.id=9f013eb15a6e4ce5abef9fe0396378b8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:43.776612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:43.778216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:43.780061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 100/204] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:43.781591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.786045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:43.787672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:43.954269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2029e66-0e9d-45b2-be65-ce1b7ee9f01e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.992982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4066e1-90ab-403d-821c-1f96589bc805 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:43.996266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:43.997125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:43.997622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:43.998097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:43.999125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, user.id=6ab36f83a70143998231c2310a7c536a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:43.999635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:44.001457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:44.008278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 101/205] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/role_assignments?user.id=6ab36f83a70143998231c2310a7c536a&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:44.049188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:44.066557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:44.067182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.067461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.067733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.171741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:44.173243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:44.287263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 105/206] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 10:42:44.292444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:44.383502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:44.420239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3911a-b21d-4412-840f-c446558bfbe2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:44.462731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:44.463377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.463660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.463981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.465136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:44.466583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a008873-f5a1-4198-92be-234be1154a3c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:44.469330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 102/207] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:44 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:44.475269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:44.509850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:44.576196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a2ea-6eec-4400-a36d-2777dccfc953 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:44.612891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:44.613503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.613782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.614086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.614893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=9f013eb15a6e4ce5abef9fe0396378b8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:44.615202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:44.616687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c0de7-7063-4846-aab8-7da4ff3ad98b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:44.627082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 106/208] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:44 2022] GET /identity/v3/role_assignments?user.id=9f013eb15a6e4ce5abef9fe0396378b8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=e0028b79d128480ba32f23616eb613c1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:44.932618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9731a-5306-4e93-89bd-97e615b1c486 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.933109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9731a-5306-4e93-89bd-97e615b1c486 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.933109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9731a-5306-4e93-89bd-97e615b1c486 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.934668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 103/209] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:44.940113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.940398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.940675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.959164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:44.960393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:44.985743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed8b17-3fcd-430e-b1cd-92854be34ea7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.986015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed8b17-3fcd-430e-b1cd-92854be34ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.986285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed8b17-3fcd-430e-b1cd-92854be34ea7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:44.987747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 104/210] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:44.993786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:44.994227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:44.994500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.016015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:45.017194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:45.052062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 107/211] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:45.069377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.092311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.102121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 105/212] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:45.127795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.198277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d7b7d-02f0-4e09-828c-721397add740 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.215253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.244838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.245473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.245747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.261341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:45.262099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18775-faa6-4364-b2b2-0af2cb28304f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:45.267365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:45.268463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:45.269260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 108/213] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:45.276650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.308691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.309857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.310364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.310852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.312704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:45.314402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:45.321010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 106/214] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:45.322054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.365896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.388462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e7366-1138-4e27-9e6e-f6373d8390f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.424639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.425484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.425594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.425976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.426688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:45.427052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:45.428530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:45.429270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33772-aff2-427e-82d4-a4016e72ffe3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.437669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 109/215] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:45.443146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.467185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.468183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.468650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.469387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.470193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:45.471868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:45.480099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.549352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cb184-2601-406e-aedc-788bdf540143 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.567569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 107/216] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:45 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:45.595460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.596165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.596491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.596769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] PATH_INFO: `/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.611048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.616973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] RBAC: Authorizing `identity:get_user(user_id=860d7fd55f5e403db96bc5e50a2bbaf8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:45.618498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:45.625329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 110/217] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:45.626843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:45.755532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1ae4-b87d-4397-844a-bbefc7140e8a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:45.794872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:45.795484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:45.796319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:45.796589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:45.857608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:45.867888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:45.869403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da25b2-6479-4111-be6c-ccc0f2907639 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:45.976265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 108/218] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:45.982397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.082754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff87d-8897-4a17-9e14-6e6d1289f1e7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.082935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff87d-8897-4a17-9e14-6e6d1289f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.083210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff87d-8897-4a17-9e14-6e6d1289f1e7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.084731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 109/219] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:46.090633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706044df-ff6e-4938-82b5-219123c1246d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.090910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706044df-ff6e-4938-82b5-219123c1246d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.091176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706044df-ff6e-4938-82b5-219123c1246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.094660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d256-5caf-4ed7-af20-eb87d4f1386c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.109700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-706044df-ff6e-4938-82b5-219123c1246d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:46.110964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-706044df-ff6e-4938-82b5-219123c1246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:46.132477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.133112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.133387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.133658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.134911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:46.136386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:46.138308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 111/220] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:45 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:46.143961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.200407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 110/221] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:46.216542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:46.223993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706044df-ff6e-4938-82b5-219123c1246d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.252190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:46.282369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1908-e8d6-4fc8-a244-44cc7a12c32c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.347033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.347560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.349670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.349949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.350686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=860d7fd55f5e403db96bc5e50a2bbaf8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:46.350994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:46.352457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:46.368453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 112/222] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:46 2022] GET /identity/v3/role_assignments?user.id=860d7fd55f5e403db96bc5e50a2bbaf8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=933643980c98414dad5713dc8820df36 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:46.394019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706044df-ff6e-4938-82b5-219123c1246d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.456639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.457302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.457577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.457846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.471474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:46.476292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:46.477169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:46.478011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 111/223] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:46.483966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.595437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832f6ae-92fc-456f-820c-312b6d190dbc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.627401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:46.633501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.634123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.634405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.634675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.635351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:46.635654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:46.637088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f186da-85eb-45cd-8f87-6b5430895e4a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:46.648087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 113/224] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:46.653754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.740182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:46.761311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0efc8-2283-4676-a4bc-1edcef880b65 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.761657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0efc8-2283-4676-a4bc-1edcef880b65 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.761847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0efc8-2283-4676-a4bc-1edcef880b65 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.763484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 114/225] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:46.767614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291ba7b-e9cd-4b88-995d-3193b5d104c0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.769568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.769853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.770124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.807173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.807828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.808106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.808368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.809489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:46.810976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:46.813236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 112/226] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:46 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:46.816768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:46.817961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:46.818759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.921545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5089272-06ce-476d-9d08-12612c38bd10 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:46.956906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:46.957582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:46.957862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:46.958129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:46.959241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:46.960876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:46.962948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 113/227] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:46 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:46.970006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:46.992163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 115/228] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:47.016126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:47.027842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:47.146891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621136a-5f46-498d-b186-bcccc169f379 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.174652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:47.187144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.187786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.188693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.188693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.189865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b752-1e18-46a9-a6e9-04c8ec867694 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.200588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:47.206444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:47.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:47.208297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 116/229] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:47.215941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:47.325775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d292301-68fe-4cf7-87f9-38c172eefc54 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.326278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.327488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.327779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.328055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/users/9f013eb15a6e4ce5abef9fe0396378b8/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.333033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, user_id=9f013eb15a6e4ce5abef9fe0396378b8, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:47.334493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:47.362674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.363281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.365923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.365923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.365923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:47.365923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:47.366437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:47.378802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 117/230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:47.381358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:47.425495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 114/231] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:46 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/users/9f013eb15a6e4ce5abef9fe0396378b8/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:47.448346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:47.515937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dda0a-da58-4c6f-9c5f-387ec0eddf1e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.553283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.553965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.554246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.554522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.555748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:47.557232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:47.558990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 118/232] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:47 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:47.565562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:47.664403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:47.675382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e15d-b09d-4182-8f2f-9941c477666e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.710826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.711528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.711829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.712100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.713236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:47.715226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6c8c3-2dec-4f78-9f6d-5c40110f6ec1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:47.722116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 115/233] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:47 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:47.723495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:47.828019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831936d8-a635-4407-a745-35bf2e77dfba None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:47.845766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:47.866172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:47.866829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:47.867112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:47.867378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/users/9f013eb15a6e4ce5abef9fe0396378b8/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:47.869563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, user_id=9f013eb15a6e4ce5abef9fe0396378b8, role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:47.871189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63456985-a43b-4173-a1ac-c23c0d8518a3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:47.976704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 119/234] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:47 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/users/9f013eb15a6e4ce5abef9fe0396378b8/roles/e0028b79d128480ba32f23616eb613c1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:48.206566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c172-59b1-4e81-92a1-7c0db7a64ad3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.207278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c172-59b1-4e81-92a1-7c0db7a64ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.207278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c172-59b1-4e81-92a1-7c0db7a64ad3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.208460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 116/235] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:48.213889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e9c63-bd9e-4bac-9271-a67ce3048310 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.214172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e9c63-bd9e-4bac-9271-a67ce3048310 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.214439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e9c63-bd9e-4bac-9271-a67ce3048310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.234993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3f1e9c63-bd9e-4bac-9271-a67ce3048310 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:48.236166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f1e9c63-bd9e-4bac-9271-a67ce3048310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:48.314635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 120/236] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:48.338254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088c172-59b1-4e81-92a1-7c0db7a64ad3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:48.467592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088c172-59b1-4e81-92a1-7c0db7a64ad3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:48.477842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2abc1c-4f1a-4156-8d61-b192a434a2f0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.478131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2abc1c-4f1a-4156-8d61-b192a434a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.478404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2abc1c-4f1a-4156-8d61-b192a434a2f0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.479911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 121/237] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:48.486950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.487170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.487442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.518406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:48.519945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:48.520165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:48.520995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.521457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.521925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.543111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:48.561975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:48.562851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 117/238] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:48.563645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:48.569124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:48.677137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 122/239] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:48.742353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993ce3-1c37-4c7f-9ee9-38860e3d97d2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:48.756192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:48.780810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:48.781424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.781753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.782030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.782701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:48.783001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:48.784424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:48.793415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 118/240] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:48.799459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:48.848816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:48.868233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:48.904213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6883-9ba0-4264-a2df-a726e7e36fa6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:48.940480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:48.941309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:48.941688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:48.942059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] PATH_INFO: `/v3/users/e978963f91904ae6b03c791f0e6a126d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:48.958446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] RBAC: Authorizing `identity:get_user(user_id=e978963f91904ae6b03c791f0e6a126d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:48.960140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:48.963286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 119/241] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:48 2022] GET /identity/v3/users/e978963f91904ae6b03c791f0e6a126d => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:48.972515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:48.976003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e1144-ab60-4106-898d-1e894036fb5b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.069124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.069686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.069965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.070234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.071358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:49.072848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00640b-f64d-41ae-8405-ac0e92206230 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:49.074683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 123/242] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:49.075839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad246-5342-4db8-bb68-0fff5ee3658d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.076125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad246-5342-4db8-bb68-0fff5ee3658d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.076399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ad246-5342-4db8-bb68-0fff5ee3658d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.082345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 124/243] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:49.083144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ad246-5342-4db8-bb68-0fff5ee3658d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:49.092325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc967003-10ee-4b8d-aa21-e6cc04bf6645 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.124282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:49.136392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.139415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.139415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.139415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.189337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:49.194424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ad246-5342-4db8-bb68-0fff5ee3658d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.226047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:49.227745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:49.229922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.230776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.231201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.231615 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.232469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:49.235473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca73c2b-a2ae-4e6c-97cd-66884f57a949 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:49.335775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 125/244] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:49 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:49.337462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.337940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.338653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.362328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 120/245] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:49.365274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:49.366705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:49.368040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:49.422911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:49.532887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa453c-457c-4ac7-bd9f-a21a4b2434d1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.536125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 126/246] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:49.574174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:49.659285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.660330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.660827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.661290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.662836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:49.664622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:49.666495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 121/247] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:49 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:49.676529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:49.719960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:49.735457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919ee0-832a-413d-883e-2812943ff379 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.788210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.788828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.789045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.789223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.811467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:49.817779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:49.820102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:49.820996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:49.821796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 127/248] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:49.833399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:49.834639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fbf2-e866-4377-ad84-5d7b5948daa6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:49.893199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:49.893733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.894008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.894276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.895009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=e978963f91904ae6b03c791f0e6a126d) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:49.895313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:49.896778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd8a7-47a9-4955-89ff-733b8dde8573 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:49.916795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 122/249] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:49 2022] GET /identity/v3/role_assignments?user.id=e978963f91904ae6b03c791f0e6a126d&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:49.919842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe0902-2631-4752-aca4-ed86293350e2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.919842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe0902-2631-4752-aca4-ed86293350e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.919842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe0902-2631-4752-aca4-ed86293350e2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.919842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 123/250] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:49.932058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:49.932359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:49.932775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:49.959524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:49.961460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:50.041991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069f0dc-21ff-4060-8d0b-d2bffc4202af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.095671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 124/251] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:50.118465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.119153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.119444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.119714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.120426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:50.120732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:50.122118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:50.141142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 128/252] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:50.147750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.147918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.171864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:50.340833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef33b50-d2bb-4b03-a1f8-fa7cdcfb1cb7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.347383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:50.368203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acceedf7-8cba-47a5-8ed4-e40c6621499b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.424220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.424778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.425056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.425327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] PATH_INFO: `/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.426447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.427074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.427353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.427613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.428695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] RBAC: Authorizing `identity:get_user(user_id=860d7fd55f5e403db96bc5e50a2bbaf8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:50.432872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:50.437325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 129/253] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:50.442783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.447415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:50.452755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:50.453671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:50.454484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 125/254] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:50.465655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.582086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48ac4d-78ac-447e-8ecd-c8ee68c1f92f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.584444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:50.604985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bb0fc-b0e2-44ea-89cc-cdc8c2e08c93 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.634302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.634911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.635186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.635632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.636321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:50.636669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:50.638166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e333a6-134e-458b-a649-c52ad309cf52 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:50.652190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 126/255] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:50.653132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.653612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7ef6-8a9c-4585-8138-835c49e7e6a2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.653791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.653884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7ef6-8a9c-4585-8138-835c49e7e6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.654049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.654145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7ef6-8a9c-4585-8138-835c49e7e6a2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.654322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.655520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 127/256] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:50.656685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:50.658276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5d73f-0cc4-461b-8019-f263f6190c65 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:50.660048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 130/257] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:50.660414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7ef6-8a9c-4585-8138-835c49e7e6a2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.661146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.661424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.661724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.690397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:50.697608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:50.702176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:50.809014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7ef6-8a9c-4585-8138-835c49e7e6a2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:50.845980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 131/258] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:50.846636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.875438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:50.876101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:50.876378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:50.876647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:50.877784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:50.879225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:50.881252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 128/259] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:50.887142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:50.987293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.024226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.034184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388301d-0f58-4412-9456-8304fabb8cd0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.069137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682ca0-af33-457a-9d5a-7c8a1d68ebed None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.095401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.096088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.096367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.096637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/users/860d7fd55f5e403db96bc5e50a2bbaf8/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.100505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, user_id=860d7fd55f5e403db96bc5e50a2bbaf8, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:51.101955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:51.121381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.121991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.122266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.123920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.163358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 132/260] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:50 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/users/860d7fd55f5e403db96bc5e50a2bbaf8/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:51.164233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:51.241902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:51.243579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:51.391107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19528b-9c23-40af-be6a-13fe83b520b8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.408693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.448605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 129/261] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 10:42:51.452828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.453462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.453740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.454012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.456346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:51.459459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.481326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:51.486607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:51.487478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:51.488309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 133/262] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:51.500636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:51.633653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa00f46-542d-40e9-ad2e-1e093674cda5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.664310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da5917-6215-4e1a-931c-91ad802e4bf2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.724339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.724965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.725242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.725513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.726180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:51.726477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:51.728072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:51.740001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 134/263] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:51.745849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:51.757162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.757742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.758019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.758288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.762721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:51.766929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:51.766929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 130/264] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:51.783270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:51.835788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.881378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:51.902425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ee9c4-ebed-425b-8121-bc1cfb943c5c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.960258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:51.967786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:51.968303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:51.969055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] PATH_INFO: `/v3/users/9f013eb15a6e4ce5abef9fe0396378b8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:51.970438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=9f013eb15a6e4ce5abef9fe0396378b8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:51.972126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:51.974219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 135/265] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/users/9f013eb15a6e4ce5abef9fe0396378b8 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:51.978066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907cecf-b23e-4d57-af36-6cfd60d0e6fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:51.979869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:52.047868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:52.047868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.047868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.047868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.048248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=9f013eb15a6e4ce5abef9fe0396378b8) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:52.048248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:52.048248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcab318-9cf9-4953-b770-585c4b02ae2e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:52.058184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 131/266] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/role_assignments?user.id=9f013eb15a6e4ce5abef9fe0396378b8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:52.159931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487db66-787c-4bf0-8f20-f9470978e2d8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:52.189030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:52.204059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:52.205561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.205561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.205561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.296234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:52.297723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:52.361410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53418-2200-459c-b2bf-2aeac64ab356 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.362010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53418-2200-459c-b2bf-2aeac64ab356 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.362478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c53418-2200-459c-b2bf-2aeac64ab356 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.364390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 132/267] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:52.369727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.370238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.370699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.420898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:52.422896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:52.478933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 136/268] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:52.484581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:52.532118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:52.580519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 133/269] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:52.633003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:52.679843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89206098-dccc-4375-933d-2d99de8d8fa4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:52.696456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:52.746378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:52.747347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.747780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.748757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.750021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:52.751678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e744-ffba-45ba-98e0-480121352b54 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:52.753986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 137/270] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:52 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:52.755567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf66-3eb6-40ee-b3a1-7020026e4d94 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.756031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf66-3eb6-40ee-b3a1-7020026e4d94 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.756442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbaf66-3eb6-40ee-b3a1-7020026e4d94 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.757948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 138/271] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:52.759752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbaf66-3eb6-40ee-b3a1-7020026e4d94 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:52.789211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbcad4-aa4f-4349-9cc2-ad2fbaaf4cd8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:52.833060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:52.833973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.834448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.834887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.846403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6cff0fe6-e271-44d6-9ec0-ccba9a7fed84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:52.852251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:52.853128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:52.853925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:52.856340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 134/272] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:52.862528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.863013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.863457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.881786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:52.882017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:52.883403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:52.892736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbaf66-3eb6-40ee-b3a1-7020026e4d94 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:52.931896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:52.932520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:52.932836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:52.933371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:52.934338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=9f013eb15a6e4ce5abef9fe0396378b8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:52.934929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:52.937185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:52.946566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 139/273] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:52 2022] GET /identity/v3/role_assignments?user.id=9f013eb15a6e4ce5abef9fe0396378b8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=e0028b79d128480ba32f23616eb613c1 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:52.947638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:52.960810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 135/274] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:52.999590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.056164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.130114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596a55b-5a46-4b8e-a2c8-21f3249245d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.176426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.176948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.177318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.177484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.178142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:53.178437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.179558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626b0f6-a85c-4598-840a-72a744500697 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.179806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2061c9-14c7-49ef-b310-3e30a98ee63e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.189151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 140/275] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.194908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a654-9b29-4330-a77a-7583460ce698 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.195199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a654-9b29-4330-a77a-7583460ce698 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.195477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a654-9b29-4330-a77a-7583460ce698 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.196773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 141/276] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:53.197670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a654-9b29-4330-a77a-7583460ce698 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.220796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.221658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.223246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.223568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.226592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.239934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-833b10d1-9479-48b9-b27a-25fc352eb259 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:53.247269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:53.248292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:53.249116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:53.251527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 136/277] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:53.252890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.253173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.253824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.280525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:53.281729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:53.333687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a654-9b29-4330-a77a-7583460ce698 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.354586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 137/278] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:53.355341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.372660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.373314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.373616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.373898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] PATH_INFO: `/v3/users/bf3e909dbf7c4acdbf897033f2285c88` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.379245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.387454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] RBAC: Authorizing `identity:get_user(user_id=bf3e909dbf7c4acdbf897033f2285c88)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.388949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.390574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 142/279] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/users/bf3e909dbf7c4acdbf897033f2285c88 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.391325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.463668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0379ff-9ba9-4e6d-a201-9613352996e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.474369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.499174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.499794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.500073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.500338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.500460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7df45-2197-4569-9df1-1beb5d47451e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.500987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:53.501281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.502647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.511816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 138/280] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.512450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.543808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.544405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.544684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.544952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.546107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.547537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.553725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 143/281] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.553725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.604121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.622465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fdfb-fe3b-4edd-b3ec-7417ed22f4fd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.652159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803675b-b68c-4d4b-bf7b-c6b6d81414ce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.656946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.657544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.657821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.658091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.658807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, user.id=bf3e909dbf7c4acdbf897033f2285c88, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:53.659121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.660583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.665506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 139/282] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/role_assignments?user.id=bf3e909dbf7c4acdbf897033f2285c88&scope.system=all&role.id=933643980c98414dad5713dc8820df36 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.666268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.694846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.695489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.695814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.696065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] PATH_INFO: `/v3/users/e978963f91904ae6b03c791f0e6a126d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.697244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e978963f91904ae6b03c791f0e6a126d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.698879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f5994-e967-4b5e-b4ff-f4576e2ac9b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.705992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 144/283] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/users/e978963f91904ae6b03c791f0e6a126d => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:53.705992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb52-77e6-44b2-beac-84b1b5adf99e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.705992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb52-77e6-44b2-beac-84b1b5adf99e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.705992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaeb52-77e6-44b2-beac-84b1b5adf99e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.705992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 145/284] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:53.706806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaeb52-77e6-44b2-beac-84b1b5adf99e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.725975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.795854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7710b-27bb-4e52-8f38-c3d1178d7742 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.825220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaeb52-77e6-44b2-beac-84b1b5adf99e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:53.837383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.838276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.838753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.839221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.840066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.841640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.863867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:53.864484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324eae3-8da2-4efa-8218-e7b85588dacf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 146/285] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:53.874173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:53.874631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:53.874631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:53.888242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:53.889427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:53.955114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 147/286] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:53.955755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:53.965348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 140/287] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:42:53 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:42:53.965880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.968937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:53.988053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:54.091727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786973a5-537f-4634-aaa3-f5fabcc4aac6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:54.135496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf289f-da27-4fb5-9b60-1559d7c521b2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:54.136097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:54.136771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:54.137890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:54.138176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/users/e978963f91904ae6b03c791f0e6a126d/roles/41f5ad14f8214ea6b35c94a515cb80fa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:54.141024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, user_id=e978963f91904ae6b03c791f0e6a126d, role_id=41f5ad14f8214ea6b35c94a515cb80fa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:54.142489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:54.176844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:54.177480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:54.177759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:54.178028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:54.189473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:54.194705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:54.198730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:54.199524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 141/288] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:54.202105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:54.226597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 148/289] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:42:53 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/users/e978963f91904ae6b03c791f0e6a126d/roles/41f5ad14f8214ea6b35c94a515cb80fa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:54.272876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:54.324971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fd3f0-71fb-4f50-9c0f-a1265c5d5620 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:54.366194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:54.366825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:54.367107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:54.367378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:54.368098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:54.368401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:54.369968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:54.380689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 142/290] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:54.386560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:54.463056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:54.494771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761549b-1da5-4957-9a04-1a1d659237b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:54.530967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:54.531561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:54.531866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:54.532353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] PATH_INFO: `/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:54.533515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=860d7fd55f5e403db96bc5e50a2bbaf8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:54.535021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:54.536975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 149/291] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:54 2022] GET /identity/v3/users/860d7fd55f5e403db96bc5e50a2bbaf8 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:54.542290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:54.665489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c43f7-35a3-4f87-963c-953bcf4a8418 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:54.696813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:54.711189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:54.711812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:54.712090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:54.712358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:54.787272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:54.788684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:54.891122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 143/292] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:54.900845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:55.057735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5fa42-4f61-4601-bfe9-4e82d545c4a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:55.125074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:55.125703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:55.125978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:55.126246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:55.127373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:55.128817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b173f48-95fb-4dc3-a3d1-4392c47085bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:55.130427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 150/293] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:54 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:55.136396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:55.160665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:55.235837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5122-a039-4162-bba8-5d251bd5b5cb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:55.269774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:55.270373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:55.270648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:55.270916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:55.271630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=860d7fd55f5e403db96bc5e50a2bbaf8) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:55.271959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:55.273437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333867e8-1ecc-4528-8ce9-5e358b362b80 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:55.279552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 144/294] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:55 2022] GET /identity/v3/role_assignments?user.id=860d7fd55f5e403db96bc5e50a2bbaf8&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=933643980c98414dad5713dc8820df36 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:55.871259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e6e-3eb1-48e9-8bdb-03a466455dca None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:55.871828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e6e-3eb1-48e9-8bdb-03a466455dca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:55.872246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247e6e-3eb1-48e9-8bdb-03a466455dca None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:55.874256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 151/295] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:55.879534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:55.880362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:55.880800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:55.899588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:55.901072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:55.986753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 145/296] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:56.008351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247e6e-3eb1-48e9-8bdb-03a466455dca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.018197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.118137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e247e6e-3eb1-48e9-8bdb-03a466455dca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.153027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.153581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.153864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.154157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.164626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:56.170456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:56.171360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:56.172266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 152/297] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:56.179059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.219017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.279657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605cf0c-15bc-47b8-902d-3ede4634b1e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.318624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.319229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.319505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.319795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.320459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:56.320758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:56.322118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a1dbd-adaf-444d-b415-95ed3a3bffd5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:56.331318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 146/298] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:56.338739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.413382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.421525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5c5c1-642c-4d1c-9cf5-7e6653e5a7e3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.421808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5c5c1-642c-4d1c-9cf5-7e6653e5a7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.422078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5c5c1-642c-4d1c-9cf5-7e6653e5a7e3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.423495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 147/299] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:56.430054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.430332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.430605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.456803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:56.458999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:56.464500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f05076-0a87-4397-948b-7b1065343edd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.500379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.501308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.501595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.501921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] PATH_INFO: `/v3/users/bf3e909dbf7c4acdbf897033f2285c88` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.503278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] RBAC: Authorizing `identity:get_user(user_id=bf3e909dbf7c4acdbf897033f2285c88)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:56.504728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:56.506808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 153/300] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/users/bf3e909dbf7c4acdbf897033f2285c88 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:56.511948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.548718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 148/301] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:56.576705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.587886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.631869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bdb48-0cd6-4db1-afd6-2df7dd644231 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.671205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.671793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.672087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.672354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] PATH_INFO: `/v3/system/users/bf3e909dbf7c4acdbf897033f2285c88/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.673777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bf3e909dbf7c4acdbf897033f2285c88, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:56.675157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:56.677630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.706063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39957b14-e210-4e87-9a5a-e6c49ab0d83f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.745986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.746616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.746892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.747225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.757720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-403aab28-d148-4364-89c5-c5e9aa64efff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:56.762406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:56.769323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 149/302] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:56.770532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f0d9-0810-436e-9163-5917ddd19638 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.771273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f0d9-0810-436e-9163-5917ddd19638 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.771273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f0d9-0810-436e-9163-5917ddd19638 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.771273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 150/303] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:56.771694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d55-322e-4e08-a6a8-d7fa80fdc706 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.771938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d55-322e-4e08-a6a8-d7fa80fdc706 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.772210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d55-322e-4e08-a6a8-d7fa80fdc706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.789521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-998a3d55-322e-4e08-a6a8-d7fa80fdc706 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:56.790718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-998a3d55-322e-4e08-a6a8-d7fa80fdc706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:56.792361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 154/304] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Aug 11 10:42:56 2022] PUT /identity/v3/system/users/bf3e909dbf7c4acdbf897033f2285c88/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:42:56.793063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.880137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:56.894914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 151/305] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:56.894914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb59df-4080-4067-a99f-5bf1855a539a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.894914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb59df-4080-4067-a99f-5bf1855a539a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.894914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb59df-4080-4067-a99f-5bf1855a539a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.894914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 152/306] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:56.897221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.898433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.898433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.919463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:56.921278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:56.924987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf47e2-abe5-4c42-8ff7-207224930fa2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:56.963065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:56.963949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:56.964376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:56.964646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:56.965310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:56.965611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:56.967038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:56.976053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 155/307] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:56.976768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:56.996421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 153/308] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:56.997084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.065056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.105085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03798144-bbc4-4a29-abc7-81d42a0597fc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.113827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaede6a3-799c-4c3b-8454-c22d6dba17e0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.133056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.141882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.142515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.142795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.143067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] PATH_INFO: `/v3/users/687cd10a36c844ec972a3d4a2e8a53f0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.149173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.149799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.150074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.150344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.156859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] RBAC: Authorizing `identity:get_user(user_id=687cd10a36c844ec972a3d4a2e8a53f0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.158323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.160017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 154/309] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/users/687cd10a36c844ec972a3d4a2e8a53f0 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:57.161089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:57.165605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:57.166330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:57.167189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 156/310] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:57.168394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.269397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b78a9-8aa0-425c-bcfb-671c9fd7fc24 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.275308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bbdf4-0770-4faa-8f60-a661134bbf9d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.290028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.303472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.304106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.304379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.304649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.305363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, user.id=687cd10a36c844ec972a3d4a2e8a53f0, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:57.305664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.307083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.312698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 157/311] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Aug 11 10:42:57 2022] GET /identity/v3/role_assignments?user.id=687cd10a36c844ec972a3d4a2e8a53f0&scope.system=all&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:57.313430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.315643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.316597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.316970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.317310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.318999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.321957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.332696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 155/312] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:57.335414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.412041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.457278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1144e4-92dc-408b-a83b-ecce996817ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.489398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be14475-57ce-442c-8436-92c5bc38e634 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.504221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.505227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.505791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.506252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.507156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:57.507648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.509494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.518420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 158/313] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:57.523884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.528662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.529368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.529653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.529920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] PATH_INFO: `/v3/groups` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.530592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.532075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.575242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.636230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1137940-25fe-4a43-b2ab-7785ef004f13 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.644940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 156/314] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Aug 11 10:42:57 2022] POST /identity/v3/groups => generated 229 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:42:57.680368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.681063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.681343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.681616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] PATH_INFO: `/v3/users/e978963f91904ae6b03c791f0e6a126d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.682848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] RBAC: Authorizing `identity:get_user(user_id=e978963f91904ae6b03c791f0e6a126d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.684380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:57.687013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 159/315] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:57 2022] GET /identity/v3/users/e978963f91904ae6b03c791f0e6a126d => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:57.696973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:57.809261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e2a-1965-4e3f-a826-69cd3d566e8b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:57.824107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:57.848413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:57.849027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:57.849303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:57.849568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:57.918779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:57.920348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:58.027200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 157/316] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:42:58.033453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:58.140638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf863e-21c8-4a9d-91af-1ec8ed1a6514 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:58.159528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:58.180372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:58.180997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:58.181277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:58.181552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:58.182672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:58.184503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef66a48-5721-4f9d-9cc5-b02c8195a370 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:58.187012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 160/317] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:42:58 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:58.192474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:58.209022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:58.298501 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c8a5e-95bc-46ce-a375-58eb2c193d66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:58.333350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:58.334299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:58.334834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:58.335300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:58.336382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, scope.project.id=05825802c74649838990b7fb930d59a4, user.id=e978963f91904ae6b03c791f0e6a126d) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:58.336890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:58.338519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5ca5e-8b14-4cc3-8253-adc68533b416 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:58.345785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 158/318] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:42:58 2022] GET /identity/v3/role_assignments?user.id=e978963f91904ae6b03c791f0e6a126d&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:58.895559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb033bf-ae3d-4992-8511-cf0faef37b69 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:58.895937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb033bf-ae3d-4992-8511-cf0faef37b69 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:58.896111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb033bf-ae3d-4992-8511-cf0faef37b69 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:58.897535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 161/319] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:58.902376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:58.902656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:58.902921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:58.921243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:58.922408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:58.989819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 159/320] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:42:59.011967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb033bf-ae3d-4992-8511-cf0faef37b69 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.109843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb033bf-ae3d-4992-8511-cf0faef37b69 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.141827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.142447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.142723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.142994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.152909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:59.157979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:59.158853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:42:59.160786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 162/321] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:42:59.165694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.176325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.266979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a678f1f-19aa-4fe5-8dc8-489ff7883c00 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.300229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.300833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.301104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.301369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.302022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:59.302320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:59.303673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88457405-6b9d-4ae6-8cdd-31810b10e9af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:59.312065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 160/322] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:59.317328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.361063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.391327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f586c-8057-493e-898f-07045e6e6fe3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.391647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f586c-8057-493e-898f-07045e6e6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.391902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f586c-8057-493e-898f-07045e6e6fe3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.393524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 161/323] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:42:59.398103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.398384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.398659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.416710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:59.417802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:59.419052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abcbd-cdac-4919-b132-608912d4d65c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.454760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.455649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.456098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.456514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] PATH_INFO: `/v3/users/bf3e909dbf7c4acdbf897033f2285c88` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.457768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] RBAC: Authorizing `identity:get_user(user_id=bf3e909dbf7c4acdbf897033f2285c88)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:59.459325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:59.461087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 163/324] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/users/bf3e909dbf7c4acdbf897033f2285c88 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:59.466659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.483792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.488485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 162/325] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:59.510371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.573155 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a45d4-6d82-4a5b-9923-e490f54dadae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.584322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.611938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e3e89-0f22-44af-b560-b4a94839eb72 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.616768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.617584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.618063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.618517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.619522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] RBAC: Adding query filter params (role.id=933643980c98414dad5713dc8820df36, user.id=bf3e909dbf7c4acdbf897033f2285c88, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:59.620058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:59.621762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:59.634995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 164/326] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/role_assignments?user.id=bf3e909dbf7c4acdbf897033f2285c88&scope.system=all&role.id=933643980c98414dad5713dc8820df36 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:42:59.656971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.657625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.657900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.658169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.671017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3b730062-6896-4663-bdbf-9c01358284ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:42:59.675788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:42:59.676726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:42:59.677560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:42:59.680096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 163/327] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:42:59.682774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.761095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bb09a-66ab-4fbf-97a6-d1f8070fd698 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.761428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bb09a-66ab-4fbf-97a6-d1f8070fd698 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.761660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bb09a-66ab-4fbf-97a6-d1f8070fd698 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.763810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 164/328] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:42:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:42:59.769220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.769523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.769765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.789890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:42:59.791431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:42:59.797084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1687f3-7cb1-4cf7-a7db-349e60d38453 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.837820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.838664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.839108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.839523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.840369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:42:59.840686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:59.842143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:59.851302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 165/329] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:42:59.857600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.864937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.876004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 165/330] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:42:59.886935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:42:59.898313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:42:59.953987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f0e77-21e5-47b0-8ec9-288a47102444 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.989033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:42:59.989648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:42:59.989917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:42:59.990190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] PATH_INFO: `/v3/users/687cd10a36c844ec972a3d4a2e8a53f0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:42:59.991290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] RBAC: Authorizing `identity:get_user(user_id=687cd10a36c844ec972a3d4a2e8a53f0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:42:59.992776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:42:59.993443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c781237-daac-4bbb-997d-c9e921172397 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:42:59.994495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 166/331] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/users/687cd10a36c844ec972a3d4a2e8a53f0 => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:00.000437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:00.027690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:00.028343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:00.028655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:00.029048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:00.030007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:00.031464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:00.033089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 166/332] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:00.039102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:00.092854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:00.095185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:00.096845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f80b8-f545-4f97-8020-f71038cccbdf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:00.129389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:00.130026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:00.130303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:00.130577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] PATH_INFO: `/v3/system/users/687cd10a36c844ec972a3d4a2e8a53f0/roles/41f5ad14f8214ea6b35c94a515cb80fa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:00.132022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=687cd10a36c844ec972a3d4a2e8a53f0, role_id=41f5ad14f8214ea6b35c94a515cb80fa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:00.133426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89b3e8-c264-4150-878f-f06a2a994008 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:00.134191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87df348-3ded-4bfd-aff1-357b3daac4e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:00.167238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:00.167853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:00.168127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:00.168392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] PATH_INFO: `/v3/groups` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:00.168994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:00.170969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621435-f7f5-4948-8d57-8589084d05b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:00.197545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 167/333] 10.222.0.48 () {58 vars in 1334 bytes} [Thu Aug 11 10:42:59 2022] PUT /identity/v3/system/users/687cd10a36c844ec972a3d4a2e8a53f0/roles/41f5ad14f8214ea6b35c94a515cb80fa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:43:00.254804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 167/334] 10.222.0.48 () {60 vars in 1134 bytes} [Thu Aug 11 10:43:00 2022] POST /identity/v3/groups => generated 226 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:02.303987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff8d4-2954-4fc3-9d32-87c38672d762 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:02.304532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff8d4-2954-4fc3-9d32-87c38672d762 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:02.304532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aff8d4-2954-4fc3-9d32-87c38672d762 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:02.306156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 168/335] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:02.311228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707be2a6-9d43-447e-a09d-da6f8486778c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:02.311643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707be2a6-9d43-447e-a09d-da6f8486778c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:02.311848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707be2a6-9d43-447e-a09d-da6f8486778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:02.334968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-707be2a6-9d43-447e-a09d-da6f8486778c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:02.336181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-707be2a6-9d43-447e-a09d-da6f8486778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:02.424308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 168/336] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:02.430302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47d21-cb8a-432e-9989-73e8ad4e6656 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:02.430586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47d21-cb8a-432e-9989-73e8ad4e6656 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:02.430860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47d21-cb8a-432e-9989-73e8ad4e6656 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:02.432477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 169/337] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:02.437855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:02.438149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:02.438419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:02.461636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:02.462812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:02.467735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df47d21-cb8a-432e-9989-73e8ad4e6656 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:02.532343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:02.582362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 169/338] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:02.623467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:02.773404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df47d21-cb8a-432e-9989-73e8ad4e6656 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:02.819275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:02.819932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:02.820208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:02.820476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:02.842750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:02.847100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:02.848239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 10:43:02.849086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:02.853326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 170/339] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:02.861555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:02.901078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4ee79-88aa-4b99-a13b-ca22898214b0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:02.920960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.000074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.001667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.001667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.001667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.033125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-558d8958-b771-4654-be38-086f3eacffb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:03.039034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:03.039944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:03.040750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:03.050075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 170/340] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:03.057245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144aa2a-4c38-498a-a09e-2e31ae6174a0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.057967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144aa2a-4c38-498a-a09e-2e31ae6174a0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.059611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144aa2a-4c38-498a-a09e-2e31ae6174a0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.063254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 171/341] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:03.065247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144aa2a-4c38-498a-a09e-2e31ae6174a0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.071475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257fc43-93a3-4256-983d-94b363fac531 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.128571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.129120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.129405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.129681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.130359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:03.130660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.132054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f472ee4-1253-41d0-a452-8aee2f43a1da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.135044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.141764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 171/342] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:03.143007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.143298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.143575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.172762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:03.174077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:03.210808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144aa2a-4c38-498a-a09e-2e31ae6174a0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.278534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.280247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.281522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.282599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.283456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:03.284331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.289220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 172/343] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:03.289719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.292028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.307077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 172/344] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:03.313234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.327845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.442549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.446679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e408c-7c3e-4872-86e5-6a95d3ef12bf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.478758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b404f-951f-4f4c-bd01-efeceab08dcc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.485160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.486109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.486633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.487098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.523230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.523924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.524203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.524472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] PATH_INFO: `/v3/users/687cd10a36c844ec972a3d4a2e8a53f0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.526429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] RBAC: Authorizing `identity:get_user(user_id=687cd10a36c844ec972a3d4a2e8a53f0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.527260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.529622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 173/345] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/users/687cd10a36c844ec972a3d4a2e8a53f0 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:03.530179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.567242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.568730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76e1d1-7b6d-49c7-a23f-58ae30cec51f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.663663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.684856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411249b-ff2a-4ad1-8bb5-55be31a59b6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.723645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:03.727309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.727965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.728345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.728518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 173/346] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:03.728813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.730008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65d16d-cd7d-4918-8361-b188127cdb54 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.730430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65d16d-cd7d-4918-8361-b188127cdb54 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.730836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65d16d-cd7d-4918-8361-b188127cdb54 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.732762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 174/347] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:03.733570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65d16d-cd7d-4918-8361-b188127cdb54 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.750132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:03.755271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:03.756427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:03.757437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:03.766227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 174/348] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:03.766227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.874035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65d16d-cd7d-4918-8361-b188127cdb54 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.902590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ad5a0-2853-4d0f-97c0-10609feb4f7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:03.914875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.915490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.915900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.916272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.917107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] RBAC: Adding query filter params (role.id=41f5ad14f8214ea6b35c94a515cb80fa, user.id=687cd10a36c844ec972a3d4a2e8a53f0, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:03.917564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.919253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73827085-dcc9-4214-a27f-2edc79672e6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.932688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 175/349] 10.222.0.48 () {56 vars in 1308 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/role_assignments?user.id=687cd10a36c844ec972a3d4a2e8a53f0&scope.system=all&role.id=41f5ad14f8214ea6b35c94a515cb80fa => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:03.933952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.934397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.940360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.958919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:03.959588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:03.959919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:03.960198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:03.961491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:03.963017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:03.969427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 175/350] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:03.970372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:03.977690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:03.981095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:03.999021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.077998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.098644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 176/351] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:04.100077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.189377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa82bcf-41d4-462f-9832-788095ffb14c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.233389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.233980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.234205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.234482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.235145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:04.235448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:04.236998 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04200c33-0e23-4e90-b893-7f2cd8cc1553 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:04.237227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d0038-6462-4eeb-8587-f28de051a267 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.248377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 176/352] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:04.249667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c690-ceca-4f82-a750-64948d5c2482 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.249957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c690-ceca-4f82-a750-64948d5c2482 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.250232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c690-ceca-4f82-a750-64948d5c2482 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.251620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 177/353] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:04.256752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406c690-ceca-4f82-a750-64948d5c2482 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.288700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.288700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.288700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.289020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.296435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:04.298618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:04.299714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 177/354] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:04.301631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.354878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.372084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.392038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406c690-ceca-4f82-a750-64948d5c2482 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.441041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.441682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.441958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.442227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.444820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818c2fc-4d4b-47b7-8971-f35e59595d81 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.455875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d99962b9-06ea-4437-8e18-b5e8fb22fa49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:04.465766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:04.467276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:04.468409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 178/355] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:04.469569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.469569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.469844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.488812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:04.489747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.490390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.490664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.490936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.491195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:04.571023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 179/356] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:04.571688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.582442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:04.584517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:04.644139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.652171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.691639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefcd71-93c0-40e6-af22-9a2a22660f04 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.709341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 178/357] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:04.710447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.732599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.733211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.734531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.734531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.734531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=933643980c98414dad5713dc8820df36, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:04.734773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:04.737184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:04.744661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 180/358] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=933643980c98414dad5713dc8820df36 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:04.745401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.871935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e0568-7f2f-4f36-90ce-0245791c34a1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.895872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356c36f-114e-4c29-aa94-f773f4de2d4b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:04.917550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.918169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.918450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.918720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.919414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:04.919721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:04.921158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d1e82-06e3-4d2f-8ef4-84f7fd928c07 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:04.928361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.931705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 179/359] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:04.932605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedc9a-3063-49f7-8a15-5e167806e8a3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.932605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedc9a-3063-49f7-8a15-5e167806e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.932809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedc9a-3063-49f7-8a15-5e167806e8a3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.934125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 180/360] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:04.935388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aedc9a-3063-49f7-8a15-5e167806e8a3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:04.939368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:04.940288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:04.942086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:04.942086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:04.953740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:04.957653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:04.958500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:04.959378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 181/361] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:04.960399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.080577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aedc9a-3063-49f7-8a15-5e167806e8a3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.087908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5e52-98d9-4b08-a856-1e7aabbac498 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.130640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.131279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.131560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.131893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.133256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.134850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac073f-d292-4911-b906-e8329154a65e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.135681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.136329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.136852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.136974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 181/362] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:05.137051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.138326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.138665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.138942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.162294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:05.163498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:05.206698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.222112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.223794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.254427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 182/363] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:05.260295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.279547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.346780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 182/364] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:05.347548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.400175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c10c-20a1-46f2-b1a5-1a8645b8761a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.443811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.449523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.450143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.450504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.450780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.451458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:05.451786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.453190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.464452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 183/365] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:05.464622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd238c-d8aa-41d8-a1b2-6bae0b1a7d8a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.464703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.506228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.506860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.507127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.507392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.508560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.509979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.513807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 183/366] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:05.514586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.555103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.591743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b3118-a0c9-4c70-b704-d77f0359a7fc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.642622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.643274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.643534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.643932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.651932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf54c3-865d-44fe-b7e5-2bc79181535e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.658368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:05.663682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:05.664787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:05.665769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 184/367] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:05.666548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.702489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.703252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.703427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.703650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.705214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.707215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.711850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 184/368] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:05.712645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.742051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.809888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f22fda-bca8-452d-9f98-69be5caea5cc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.851289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.852493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.852493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.852493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.854712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfc0fd-e0e8-491b-829b-6ada39e3af36 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:05.855493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:05.898095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.928261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:05.928725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 185/369] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=e0028b79d128480ba32f23616eb613c1 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:05.928725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:05.952524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:05.954119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.039690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.059532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca3f81-924b-4a29-b231-60aa44ea1cd3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.083224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 185/370] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:06.084481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.104695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.105310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.105641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.105867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.106584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:06.106891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.108301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.122826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 186/371] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.123682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.126399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.206353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a04d9c-fa2c-4d3a-be90-c9f2035b16d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.244437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032491a-f4da-4a51-8bf4-fb01963b2729 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.245491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2524-97a5-4096-a363-19c001717738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.246276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2524-97a5-4096-a363-19c001717738 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.246643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2524-97a5-4096-a363-19c001717738 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.246926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2524-97a5-4096-a363-19c001717738 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.248112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c2524-97a5-4096-a363-19c001717738 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.249636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c2524-97a5-4096-a363-19c001717738 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.251649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 186/372] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.252517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2524-97a5-4096-a363-19c001717738 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.293525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.294231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.294514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.294788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.296812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.298363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.300474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 187/373] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.301410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.333389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.364390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2524-97a5-4096-a363-19c001717738 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.365919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.407003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.407848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.408133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.408404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.436647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa6b7e-77a9-4b25-a09d-5cd4ed89bc22 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.477032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.477692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.477958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.478248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.479093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=933643980c98414dad5713dc8820df36, scope.project.id=05825802c74649838990b7fb930d59a4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:06.479413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.480961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.490655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 188/374] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=933643980c98414dad5713dc8820df36 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.491772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.496644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.498612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.617907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.640927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2308cda-bbf7-4f0c-bc84-54a7fcdea7c3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.649410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 187/375] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:06.655272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.657776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:06.692581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.693222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.693503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.693778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.696072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.697871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.699523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 189/376] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.707528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.829330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db704bcc-a057-4fee-8805-b6fd9971fee3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.831624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fd7b1-019e-4377-8204-c0e0f3e7f1a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:06.871012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.872561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.873051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.873913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.874850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=05825802c74649838990b7fb930d59a4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:06.875362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.877534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.884349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 190/377] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=e0028b79d128480ba32f23616eb613c1 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.912838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:06.913487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:06.913769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:06.914040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] PATH_INFO: `/v3/projects/345ef69aeca14be9ac484aeebec744ed` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:06.915706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] RBAC: Authorizing `identity:get_project(project_id=345ef69aeca14be9ac484aeebec744ed)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:06.917188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc9dff-2f49-490d-a3d1-1cd076ef1dcc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:06.918963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 188/378] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/projects/345ef69aeca14be9ac484aeebec744ed => generated 344 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:06.924864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:06.949250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:07.039740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d544c-714d-4e6d-a149-dd858622135f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.040133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d544c-714d-4e6d-a149-dd858622135f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.040420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d544c-714d-4e6d-a149-dd858622135f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.041786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 189/379] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:07.047493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.047818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.048082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.067915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fd13b-c1ab-4ede-ae38-335bed691648 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:07.092244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:07.093713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:07.106305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:07.107418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.107920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.108402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] PATH_INFO: `/v3/groups/77a9e09d705149a1959e6bd95d8965fe` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.115095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] RBAC: Authorizing `identity:get_group(group_id=77a9e09d705149a1959e6bd95d8965fe)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:07.117005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:07.119216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 191/380] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:06 2022] GET /identity/v3/groups/77a9e09d705149a1959e6bd95d8965fe => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:07.129614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:07.211838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:07.223431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 190/381] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:07.236782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:07.259797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5390b1e-57d5-4141-86f5-d260ec5024fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:07.265168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:07.324938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:07.326098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.326380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.326655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.327447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] RBAC: Adding query filter params (group.id=77a9e09d705149a1959e6bd95d8965fe, role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=345ef69aeca14be9ac484aeebec744ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:07.327784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:07.329208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:07.342109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 192/382] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:07 2022] GET /identity/v3/role_assignments?group.id=77a9e09d705149a1959e6bd95d8965fe&scope.project.id=345ef69aeca14be9ac484aeebec744ed&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:07.409699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e474-c24a-4ef3-b593-9085d2addac5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:07.451178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:07.451814 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.452101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.452369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.466559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:07.471699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:07.472622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:07.473840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 191/383] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:07.478769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:07.611299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f1c4-90ec-45be-a6a2-8ac2bd749ef3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:07.654922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:07.655549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.655852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.656126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.656857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:07.657152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:07.658565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:07.673186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 193/384] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:07.678229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:07.774307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:07.820095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c486b2-bf73-4d12-a11f-72f13f6cad11 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:07.865050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:07.865699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:07.865981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:07.866251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:07.867396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:07.868883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:07.870769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 192/385] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:07 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:07.876435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:07.882519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:08.006319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608279-52b8-4b88-a56d-f626fc70abb4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:08.049669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:08.050702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.051173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.051647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.055004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:08.056667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61b351-d129-4a67-867d-51567910da0c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:08.058818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 194/386] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:07 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:08.064319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:08.210424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da895585-46d9-4a85-981d-139ff8f10717 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:08.237946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:08.245695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:08.246359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.246639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.246970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.248934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, group_id=efb8d3a0156e4ff7a5eb715944dd427f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:08.254241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ece32f-7b52-4094-8d1b-be3bca54e8a1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:08.341989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 193/387] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Aug 11 10:43:08 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:08.482573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ff94-9c96-4e07-8934-af51da87395d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.483578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ff94-9c96-4e07-8934-af51da87395d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.484120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ff94-9c96-4e07-8934-af51da87395d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.486073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 195/388] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:08.491344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.491842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.491920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.520952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:08.522197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:08.609027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 194/389] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:08.632569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ff94-9c96-4e07-8934-af51da87395d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:08.725371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:08.766491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ff94-9c96-4e07-8934-af51da87395d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:08.812389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:08.813021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.813291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.813555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.829492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:08.835030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:08.836033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:08.836859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:08.838981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 196/390] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:08.844362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:08.952227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971ab11-ef2c-4820-a434-3e6ae4fcb99d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:08.970391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:08.987517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:08.988161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:08.988442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:08.988716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:08.989380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:08.989681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:08.991060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:08.999593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 195/391] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:09.005164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:09.128504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c7f75-de99-4714-a567-b18032260305 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:09.175089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:09.185359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:09.192791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.192791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.192791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.192791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:09.193918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:09.195507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 197/392] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:09 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:09.201155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:09.334600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464a979-8e9c-4f35-970e-786bc3cbb37e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:09.385670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:09.386353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.386637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.386911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.391626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:09.393177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a0146-1b87-44dc-86e3-873aca232fad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:09.395060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 196/393] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:09 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:09.400619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:09.538736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fa5aa-6dde-4c53-8b1b-853c45e7aa70 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:09.570739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bbb8-e17f-4daa-8e06-02ecc246b01c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.571275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bbb8-e17f-4daa-8e06-02ecc246b01c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.571275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85bbb8-e17f-4daa-8e06-02ecc246b01c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.572657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 197/394] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:09.583358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.583649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.583959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.597452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:09.597452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.597452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.597452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.597826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb, group_id=efb8d3a0156e4ff7a5eb715944dd427f, role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:09.597826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d43ed6-f95c-45a7-87fa-5af44d474ff5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:09.612139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:09.613513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:09.659967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:09.684889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 198/395] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:09.706562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:09.738267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 198/396] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Aug 11 10:43:09 2022] PUT /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/e0028b79d128480ba32f23616eb613c1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:09.755561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de95e3-e255-4b08-ae1e-a066fa30d497 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.756330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de95e3-e255-4b08-ae1e-a066fa30d497 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.756816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de95e3-e255-4b08-ae1e-a066fa30d497 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.758683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 199/397] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:09.765175 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6eca6-e1e4-46db-b0c0-a04c80c9a40f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.765479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6eca6-e1e4-46db-b0c0-a04c80c9a40f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.765740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6eca6-e1e4-46db-b0c0-a04c80c9a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.800864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f9b6eca6-e1e4-46db-b0c0-a04c80c9a40f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:09.802127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f9b6eca6-e1e4-46db-b0c0-a04c80c9a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:09.823118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4141c38-85ee-48b6-8e09-0462f27f12b3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:09.861440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:09.862008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.862285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.862557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.875568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6145032e-6afb-4827-b861-12c4fb1a87b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:09.880225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:09.881144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:09.882536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 199/398] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:09.883195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13f9b-0c7d-4be4-ace9-7856f0d207eb None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.883261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13f9b-0c7d-4be4-ace9-7856f0d207eb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.883520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13f9b-0c7d-4be4-ace9-7856f0d207eb None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.885154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 200/399] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:09.887044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13f9b-0c7d-4be4-ace9-7856f0d207eb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:09.914337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:09.929754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 200/400] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:09.931084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:09.931366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:09.931637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:09.936024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:09.952506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:09.953400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:10.015950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13f9b-0c7d-4be4-ace9-7856f0d207eb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.039923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 201/401] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:10.040885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.069589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.070201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.070478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.070747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.071411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:10.071712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.073115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.083201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 201/402] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:10.084076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.183839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbe7db-effa-40fc-8483-046d9573064c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.191962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.195219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e18df-5f74-4c33-880b-e6f68ce528c7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.208426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.231794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.232915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.233610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.234126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.234547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.235266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.235616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.235987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.247837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:10.248345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:10.253444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:10.254310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:10.255318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:10.256232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:10.257236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:10.258224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:10.258965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 202/403] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:10.258965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.263852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 202/404] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:10.267449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.378120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c5fb-25aa-4f29-95b5-fc79898061e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.405210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ebd94-7302-4692-8efd-ec5dd0388946 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.414916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.415611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.415926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.416203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.417422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.419498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.421551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 203/405] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.422320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.446241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.446893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.447173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.447442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.448785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:10.448785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.451571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.452290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.460860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 203/406] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.467565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.506419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.530529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b484a-8e16-4416-8f12-b4f1e88d4731 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.571253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ff26-af52-4ffd-8d2b-6634d45d93c3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.573449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.574633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.575106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.575523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.576368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:10.576852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.578491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.594130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 204/407] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.595347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.612330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.612995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.613438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.613619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.614768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.616369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.623080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 204/408] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.624407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.699713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.709906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9d2e-3f2f-42b5-a353-ec8a55a9a493 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.733966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e1556-7f45-4617-b0c9-5c7d2d730508 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.744757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.745392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.745671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:10.745772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.745946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.747958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.749408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.755391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 205/409] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.755391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.776870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.777512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.777793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.778107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] PATH_INFO: `/v3/groups/77a9e09d705149a1959e6bd95d8965fe` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.779214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] RBAC: Authorizing `identity:get_group(group_id=77a9e09d705149a1959e6bd95d8965fe)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.780659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e1121-faa7-454e-ac48-b7f937da3fc3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 205/410] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/groups/77a9e09d705149a1959e6bd95d8965fe => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751a866-1917-440d-9b45-0b9c38bf72d0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751a866-1917-440d-9b45-0b9c38bf72d0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751a866-1917-440d-9b45-0b9c38bf72d0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 206/411] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:10.786791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751a866-1917-440d-9b45-0b9c38bf72d0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.866363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fdc98-2f3f-4ce2-bdc2-879e652d7e83 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.896421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751a866-1917-440d-9b45-0b9c38bf72d0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:10.902056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.902711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.903133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.903407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.905626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, group_id=efb8d3a0156e4ff7a5eb715944dd427f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.907169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f5539-5c5e-4354-9b8a-e4b422b18dd2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.929785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:10.930402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:10.930677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:10.930944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:10.932062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:10.933612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:10.935728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 207/412] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:10.936483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:10.985914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:11.019612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 206/413] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Aug 11 10:43:10 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:11.019612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.019612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.019612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.036797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:11.038394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:11.055053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:11.062487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0835f-7f0c-4592-8ce6-e22fa4dd8418 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.108724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.109836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.110253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.110700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] PATH_INFO: `/v3/projects/345ef69aeca14be9ac484aeebec744ed` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.112395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=345ef69aeca14be9ac484aeebec744ed)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:11.113978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:11.116048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 208/414] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:10 2022] GET /identity/v3/projects/345ef69aeca14be9ac484aeebec744ed => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:11.117036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:11.126580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 207/415] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:11.127938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:11.242305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c20b23-f912-4849-a26f-fb8367fd51b2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.259558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841dd75-7601-4473-ba3b-62da2036e2c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.277347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:11.279748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.280589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.281022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.281442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] PATH_INFO: `/v3/projects/345ef69aeca14be9ac484aeebec744ed/groups/77a9e09d705149a1959e6bd95d8965fe/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.283564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] RBAC: Authorizing `identity:create_grant(project_id=345ef69aeca14be9ac484aeebec744ed, group_id=77a9e09d705149a1959e6bd95d8965fe, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:11.285188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:11.301271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.302009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.302407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.302758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.305053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] RBAC: Authorizing `identity:create_grant(project_id=05825802c74649838990b7fb930d59a4, group_id=efb8d3a0156e4ff7a5eb715944dd427f, role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:11.306609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:11.387986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 208/416] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Aug 11 10:43:11 2022] PUT /identity/v3/projects/345ef69aeca14be9ac484aeebec744ed/groups/77a9e09d705149a1959e6bd95d8965fe/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:11.389406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:11.407077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:11.416515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 209/417] 10.222.0.48 () {58 vars in 1442 bytes} [Thu Aug 11 10:43:11 2022] PUT /identity/v3/projects/05825802c74649838990b7fb930d59a4/groups/efb8d3a0156e4ff7a5eb715944dd427f/roles/e0028b79d128480ba32f23616eb613c1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:11.527324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264f1dc-7349-45fc-973a-0c73bec88ce6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.576695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.578215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.578710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.579204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.604547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:11.610906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:11.611894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:11.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:11.615310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 209/418] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:11.622445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:11.748857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db587c-9f9f-4387-a822-698691984666 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.797102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.797635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.797918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.798190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:11.798856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:11.799161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:11.800551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525c5af-8b92-4dd2-bbb4-e8fe379b664d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:11.809373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 210/419] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:11.816873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:11.906523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:11.947616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24337ca9-6cd0-4fc7-9a9f-76054c1e49b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:11.993524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:11.994532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:11.995059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:11.995553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.026099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705db1-7a5f-4926-96af-c53902078968 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.026426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705db1-7a5f-4926-96af-c53902078968 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.026661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705db1-7a5f-4926-96af-c53902078968 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.028330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 211/420] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:12.033164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.033447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.033718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.054193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:12.055303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:12.083674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:12.108553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:12.110294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:12.128795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 212/421] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:12.160374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.213570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:12.264979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 210/422] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:12.268172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.329346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49220f0-8b18-4168-ae7c-72f65d94c5bd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.367014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ec3b6-874c-4a56-b293-773364889b04 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.374141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:12.375021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.375502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.376033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.387095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:12.392846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:12.393783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:12.394599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:12.397372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 213/423] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:12.403685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.441173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:12.446419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:12.447065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.447345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.447620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.452454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:12.453915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:12.455559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 211/424] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:12.461706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.511271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d5bb-bcf4-4584-8ff3-f115ae6db02f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.538881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:12.549237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:12.549999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.550377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.550761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.551600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:12.551994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:12.553444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:12.567507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 214/425] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:12.576261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.666069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1bfb3-d4f6-4b81-a15a-0b730699eda5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.687354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529796cb-b8c1-4f71-9d09-6bcfca41c301 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.707998 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:12.723353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:12.724222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.724909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.725816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.729741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:12.731514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:12.733770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:12.734198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:12.735515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:12.740010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:12.744099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 212/426] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:12.748265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:12.866374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:12.868054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:12.932517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89938340-394c-41ef-89ff-9a2c76836341 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:12.940206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.009746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.010627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.011161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.016134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.016906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=933643980c98414dad5713dc8820df36, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:13.017226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:13.018669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:13.022866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.034187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 213/427] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=933643980c98414dad5713dc8820df36 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:13.040160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 215/428] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 10:43:13.046752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.162317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851cff7-625b-4917-8931-b4f96eb38ed4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:13.201174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.202067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.202489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.202905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] PATH_INFO: `/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.204287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] RBAC: Authorizing `identity:get_project(project_id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:13.205966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:13.208140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 214/429] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/projects/f6bdb02c7fbf4bf6b8b2af2fc286ccdb => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:13.213954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.340337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6fbc-9dc1-46ff-b362-51e368c370af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:13.377287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.378154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.378623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.379039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.380320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:13.381902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914d58c-3015-4937-971b-a9e9b2861aa4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:13.383929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 216/430] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:13.390543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.404208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a000-fd40-423b-9cb4-e11f3ae2a170 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.406818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a000-fd40-423b-9cb4-e11f3ae2a170 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.407081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a000-fd40-423b-9cb4-e11f3ae2a170 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.408825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 217/431] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:13.414638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.414874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.415124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.440842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.442756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:13.444191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:13.502427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.517193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3084d-43cb-4ef5-8f66-095d0382073c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:13.523298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 218/432] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:13.548532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.561858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.562772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.563247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.563716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.564741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:13.565279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:13.567069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:13.575256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 215/433] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=f6bdb02c7fbf4bf6b8b2af2fc286ccdb&role.id=e0028b79d128480ba32f23616eb613c1 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:13.682419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437718c8-21ca-404f-b30a-2d44904cfad5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:13.723034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.723670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.723961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.724232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.737432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ce28408c-cbd6-41d0-8be1-37a5570bbd49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:13.742430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:13.743394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 10:43:13.744249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:13.747122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 219/434] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:13.750564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.823591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d4a80-bc46-45a2-a7f0-4d3fbc9fd678 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.824101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d4a80-bc46-45a2-a7f0-4d3fbc9fd678 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.824388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d4a80-bc46-45a2-a7f0-4d3fbc9fd678 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.826781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 220/435] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:13.834969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.835250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.835931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.863244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:13.863244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:13.872504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.875846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715b9f9-8ce1-418b-bf1e-5a1450f9d558 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:13.916423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:13.925760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:13.926267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:13.926550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:13.927044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:13.927551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:13.928012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:13.929467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:13.940513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 216/436] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:13.947003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:13.951311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 221/437] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:13.974310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.073733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd5c5c-860e-437a-9278-6d24a8440a44 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.117274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e0ec-c8c7-43cf-ac1e-49d0029a6638 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.129340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.129954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.130286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.130504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.161686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.162343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.162625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.162909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.170284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.175997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:14.181256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:14.185655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:14.186462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 222/438] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:14.189540 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.212854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.214382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe665677-6bd7-4082-87fe-0a08ad7b6563 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.232924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.303997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46011-bfa7-4501-83ee-292313b8ed50 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.322927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 217/439] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 10:43:14.322927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8442-b2e8-484c-8735-310d246fb201 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.322927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8442-b2e8-484c-8735-310d246fb201 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.322927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe8442-b2e8-484c-8735-310d246fb201 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.322927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 218/440] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:14.325143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe8442-b2e8-484c-8735-310d246fb201 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.337553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.338144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.338419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.338694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.339345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:14.339641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.341103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292e490-ada8-44d0-b908-f500d707b42f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.349063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 223/441] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:14.350169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.350447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.350712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.367132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:14.368331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:14.390343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.429400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe8442-b2e8-484c-8735-310d246fb201 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.433536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 224/442] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:14.434719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.465887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.466469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.466786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.467092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.468471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.469623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.469906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.471532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 219/443] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:14.472319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.546028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f2ce7-f9ef-4ab5-a85b-008c48932f1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.576623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd14178-e85f-4d26-827b-fbc529ea9530 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.578457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.579051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.579324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.579590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.608039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.613824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.618488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.618488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.618488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.625074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:14.629725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:14.630707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 220/444] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:14.631519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.648313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.649770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.676746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.735067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d29c4-1b5d-4e98-b856-2e85b26eaae6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.747815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 225/445] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:14.748708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.769540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.770155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.770428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.770699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.771818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.773275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.774944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 221/446] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:14.775540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.832071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.851096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d7ef-1cd3-4e6a-a3c1-ed2bc21ef27a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.872264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e2bb6-2003-4f50-87dc-e3a97ba16300 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:14.875938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:14.884615 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.885213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.885494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.885768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.886419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:14.887547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.888665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.896384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 226/447] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:14.898409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.906055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:14.906670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:14.906941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:14.907207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] PATH_INFO: `/v3/projects/05825802c74649838990b7fb930d59a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:14.908329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] RBAC: Authorizing `identity:get_project(project_id=05825802c74649838990b7fb930d59a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:14.909809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:14.911407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 222/448] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/projects/05825802c74649838990b7fb930d59a4 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:14.912172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:14.995547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e5a7c-500e-44ca-827f-43c4b23cbb75 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.008807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498ae52-83fa-4632-9746-9398b15b9d12 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.030233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.030822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.031094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.031358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.032098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=933643980c98414dad5713dc8820df36, scope.project.id=05825802c74649838990b7fb930d59a4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:15.032399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.033821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.034954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:15.039730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 227/449] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=933643980c98414dad5713dc8820df36 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:15.040504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:15.043857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.044472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.044751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.045023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.110584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:15.124074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.125919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.164514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7d49-792c-46b5-845a-56e272155b65 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.205177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.205811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.206095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.206370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] PATH_INFO: `/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.207495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] RBAC: Authorizing `identity:get_group(group_id=efb8d3a0156e4ff7a5eb715944dd427f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.209031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.210767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 228/450] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:15 2022] GET /identity/v3/groups/efb8d3a0156e4ff7a5eb715944dd427f => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:15.217395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:15.240894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 223/451] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:15.248400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:15.278429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:15.330095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521f626-533d-4876-9c3c-0fab5c43d0dd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.352684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:15.368663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b357a-07d2-4204-bba5-e3ee1a6e3b6c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.369625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.370242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.370528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.370803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.372137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] RBAC: Adding query filter params (group.id=efb8d3a0156e4ff7a5eb715944dd427f, role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=05825802c74649838990b7fb930d59a4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:15.376203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.377651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.385218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 229/452] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:15 2022] GET /identity/v3/role_assignments?group.id=efb8d3a0156e4ff7a5eb715944dd427f&scope.project.id=05825802c74649838990b7fb930d59a4&role.id=e0028b79d128480ba32f23616eb613c1 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:15.411353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.412475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.412797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.413080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] PATH_INFO: `/v3/projects/345ef69aeca14be9ac484aeebec744ed` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.414759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=345ef69aeca14be9ac484aeebec744ed)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.417552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.421576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 224/453] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:43:15 2022] GET /identity/v3/projects/345ef69aeca14be9ac484aeebec744ed => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:15.427748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:15.530731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5a8f3-18b5-4c0c-a3d6-c6254e50f487 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.565803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.566724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.567014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.567286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] PATH_INFO: `/v3/groups/77a9e09d705149a1959e6bd95d8965fe` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.568496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] RBAC: Authorizing `identity:get_group(group_id=77a9e09d705149a1959e6bd95d8965fe)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.569945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcf1cc-cef0-46bf-8bf2-97d474089b20 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.571564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 230/454] 10.222.0.48 () {56 vars in 1180 bytes} [Thu Aug 11 10:43:15 2022] GET /identity/v3/groups/77a9e09d705149a1959e6bd95d8965fe => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:15.577881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:15.672500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:15.678993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f16bd9-c2c8-4f5a-ad04-a21b829adbb3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:15.712277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:15.712877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:15.713153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:15.713422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:15.714144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] RBAC: Adding query filter params (group.id=77a9e09d705149a1959e6bd95d8965fe, role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=345ef69aeca14be9ac484aeebec744ed) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:15.714959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:15.716458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b77a03-3f92-41e6-8bc4-8f8ddb649dca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:15.723960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 225/455] 10.222.0.48 () {56 vars in 1376 bytes} [Thu Aug 11 10:43:15 2022] GET /identity/v3/role_assignments?group.id=77a9e09d705149a1959e6bd95d8965fe&scope.project.id=345ef69aeca14be9ac484aeebec744ed&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:18.065548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca98d2-e2ba-47e0-bfb5-b48261aa708e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.066406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca98d2-e2ba-47e0-bfb5-b48261aa708e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.066863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca98d2-e2ba-47e0-bfb5-b48261aa708e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.068427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 231/456] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:18.074498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e1-e91a-4a13-a27e-5e1622260059 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.075383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e1-e91a-4a13-a27e-5e1622260059 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.075895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58d7e1-e91a-4a13-a27e-5e1622260059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.095666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aa58d7e1-e91a-4a13-a27e-5e1622260059 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:18.097228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa58d7e1-e91a-4a13-a27e-5e1622260059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:18.189386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 226/457] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:18.229169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca98d2-e2ba-47e0-bfb5-b48261aa708e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:18.312870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:18.324744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffa4a-6907-4f65-89ff-60793c5fafb9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.324997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffa4a-6907-4f65-89ff-60793c5fafb9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.325223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffa4a-6907-4f65-89ff-60793c5fafb9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.327030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 227/458] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:18.336643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.337164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.337629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.366535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:18.371337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:18.393872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca98d2-e2ba-47e0-bfb5-b48261aa708e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:18.456527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:18.457157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.457432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.457701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.476282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:18.477722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:18.497271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 228/459] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-57e08746-c3ee-4105-8d92-32c85f1ba2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:18.509357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:18.510328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:18.511187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:18.511187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:18.511187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:18.511187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 232/460] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:18.516760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:18.540459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:18.552052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab4-ae89-451d-94e3-002bdd22dfe2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.552542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab4-ae89-451d-94e3-002bdd22dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.552988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab4-ae89-451d-94e3-002bdd22dfe2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.554689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 233/461] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:18.557665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62cab4-ae89-451d-94e3-002bdd22dfe2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:18.648060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9263a1-5a28-458c-8d8f-c6699cbd26ae None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:18.680203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62cab4-ae89-451d-94e3-002bdd22dfe2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:18.682810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:18.685255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:18.685867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.686146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.686416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.687073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:18.687373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:18.688767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fe99a-55c9-457b-800f-dfe302980559 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:18.707612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 229/462] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:18.707612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.707612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.707612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.724605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:18.725561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.725689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:18.726185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.726699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.726809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:18.739268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:18.741104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:18.756358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:18.757292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:18.758148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:18.758148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:18.758148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:18.760297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 234/463] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:18.761464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:18.817403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 230/464] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:18.818813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:18.829898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:18.886741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd99c-7bf9-4687-858f-96c71a35889d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:18.937696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:18.938311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.938590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.938871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.939503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:18.940957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114185a-d1cb-4c6b-87a6-383ebf28cbf3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:18.941629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500a6ad-3ab9-4c68-9ad9-415d0c98ff6e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:18.981068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:18.982032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:18.982574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:18.983085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:18.983986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:18.984479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:18.986208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:18.998383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 231/465] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:18.999495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.019112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 235/466] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:18 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:19.020560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e9f0e-8269-44bd-a181-b255d40bd47f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.020928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e9f0e-8269-44bd-a181-b255d40bd47f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.021207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e9f0e-8269-44bd-a181-b255d40bd47f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.022522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 236/467] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:19.035824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e9f0e-8269-44bd-a181-b255d40bd47f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.051396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.096650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.142282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10215831-7e31-425f-a1d0-f1e8730f213c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.151280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e9f0e-8269-44bd-a181-b255d40bd47f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.189390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.190232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.190607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.191080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.191448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060246d-b403-407e-8222-c55460586413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.192268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060246d-b403-407e-8222-c55460586413 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.192546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060246d-b403-407e-8222-c55460586413 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.192815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060246d-b403-407e-8222-c55460586413 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.193569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060246d-b403-407e-8222-c55460586413 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.195223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060246d-b403-407e-8222-c55460586413 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.204637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.207075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6eb5320d-10f7-4faf-a56c-84c217c06e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:19.222506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:19.223480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:19.224429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:19.224429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:19.224429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:19.224429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 232/468] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:19.225065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.225158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.226096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.247403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:19.248750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:19.272717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 237/469] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:19 2022] POST /identity/v3/users => generated 248 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:19.273908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060246d-b403-407e-8222-c55460586413 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.353850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 233/470] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:19.367502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.380091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.407850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060246d-b403-407e-8222-c55460586413 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.432892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.445979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.446595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.446875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.447143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.447813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:19.448122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.450295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.463974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 238/471] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:19.470085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.513209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1615977-e40c-4615-b8b1-604160cf27ec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.557197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.558709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.558961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.559233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.570702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.573089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:19.591243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:19.592216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:19.593920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:19.593920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:19.593920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:19.593920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 234/472] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:19.599194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.631221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1677a-9b7f-4435-ba1e-5e07cc4df6b1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.667075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.677742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.678357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.678634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.679165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.679848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.682184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.727297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ef289-de35-4b72-8a70-8d9b5f26da5e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.729245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:19.764704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.765337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.765680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.765954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.766600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:19.766901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.768343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd39f-addb-4b37-85e0-dd16531d08f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:19.784995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 239/473] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:19 2022] POST /identity/v3/users => generated 248 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:43:19.788449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 235/474] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:19.794297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:19.917167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42645-3aef-4099-bdd6-972b5e84b763 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:19.918077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a159b-d4e6-4baf-a686-3c86cdf805a5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.918369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a159b-d4e6-4baf-a686-3c86cdf805a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.918636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a159b-d4e6-4baf-a686-3c86cdf805a5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.920314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 236/475] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:19.926961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.927358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.927515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.950630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:19.951871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:19.982891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:19.983518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:19.983809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:19.984086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:19.984747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:19.986202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:20.053759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 237/476] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:20.079269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:20.100980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 240/477] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:19 2022] POST /identity/v3/users => generated 249 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:20.188004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:20.198585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8015b-607c-4d0a-8789-0a4ee1eb9702 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:20.238601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:20.239233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:20.239511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:20.239791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:20.240489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:20.241931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:20.286323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 10:43:20.286323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 10:43:20.286323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [parameters: {'id': 'f9527bf42e4a472cab1c57458cbce606', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 10:43:20.286323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 11 10:43:20.302362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return ref.to_dict() Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:43:20.303304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 10:43:20.304260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 10:43:20.305135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:43:20.306063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:43:20.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': 'f9527bf42e4a472cab1c57458cbce606', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:20.308030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 10:43:20.309005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:20.310033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 238/478] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Aug 11 10:43:20 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:20.310943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:20.376212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:20.463614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d069bb-c7a2-4d66-be3e-265a5ea132fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:20.508532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:20.509163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:20.509438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:20.509708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:20.532650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d9275353-a4ed-43dd-b1f4-7cf388d9bf9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:20.537950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:20.538841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:20.540226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 241/479] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:20.545041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:20.721119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651035a8-a7a3-4f45-b109-e2663653328e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:20.762749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:20.763354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:20.763631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:20.763932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:20.764597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:20.764901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:20.766268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be719a2-5f60-40c7-b2bf-b45093edde3b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:20.777813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 239/480] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:21.575793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21759df-efda-4506-b46c-c513392f09ca None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:21.576310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21759df-efda-4506-b46c-c513392f09ca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:21.576528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21759df-efda-4506-b46c-c513392f09ca None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:21.578137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 242/481] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:21.583401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a588c-aaa1-497b-9334-70cae4f7bf84 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:21.583821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a588c-aaa1-497b-9334-70cae4f7bf84 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:21.583991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a588c-aaa1-497b-9334-70cae4f7bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:21.613131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e14a588c-aaa1-497b-9334-70cae4f7bf84 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:21.614358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e14a588c-aaa1-497b-9334-70cae4f7bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:21.631020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:21.707863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 240/482] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:21.728342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21759df-efda-4506-b46c-c513392f09ca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:21.855876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:21.860282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21759df-efda-4506-b46c-c513392f09ca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:21.873430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562504-d015-4e75-b54c-e37c53be71b3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:21.873701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562504-d015-4e75-b54c-e37c53be71b3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:21.873967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c562504-d015-4e75-b54c-e37c53be71b3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:21.875538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 241/483] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:21.880227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:21.880755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:21.881262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:21.900944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:21.901558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:21.901838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:21.902626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:21.913211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:21.917864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:21.918765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:21.919569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:21.921682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:21.923072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:21.923803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 243/484] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:21.924985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.015937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 242/485] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:22.032427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff99673-c2ce-4f6d-8014-f2ca57069846 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.049120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.068378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.068986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.069275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.069510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.070167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:22.070466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.072005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.081033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 244/486] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:22.086381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.112414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.182500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.194342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ab954-8dd9-42ef-ac79-0de67991d50a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.223433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bfe02-92e4-423b-beb1-433c83ed98c9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.231355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.232008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.232282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.232546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.254941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.256453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:22.283482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:22.284460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:22.285397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:22.285397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:22.285397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:22.286988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 245/487] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:22.289981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.290673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.290948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.291217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.299583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.315838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:22.327146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:22.329249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:22.331517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 243/488] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:22.340929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.404199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.431662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913325ce-ec20-4472-9832-f4be69517947 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.468730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.469468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.469743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.470011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.470657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:22.470956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.472525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.487087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 246/489] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:22.493144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.575961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14b92-a0f4-4999-a41d-fcd8a7c66796 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.619150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.621134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.621639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.622117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.623018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:22.623518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.625102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a07f85-2b5d-431f-b69c-168e1b9c9798 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.630520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.634215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 244/490] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:22.635638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6bbe-382d-4c49-ad33-93fcf075b9a6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.636262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6bbe-382d-4c49-ad33-93fcf075b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.636771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6bbe-382d-4c49-ad33-93fcf075b9a6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.642144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 245/491] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:22.643102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6bbe-382d-4c49-ad33-93fcf075b9a6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.647379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60daa-2c81-4361-8994-cf4edbd9b9e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.685255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.686209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.686685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.687178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.689550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.695388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.697034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-03f5bbdd-3796-4c99-8d27-ee6da96835ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:22.711133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:22.712323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:22.713463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:22.715248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 247/492] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:22.716624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.717006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.717621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.737677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:22.738865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:22.758810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6bbe-382d-4c49-ad33-93fcf075b9a6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.792648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.793259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.793532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.793797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.804427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.805949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.806611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 248/493] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:22.809151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d270ec73-845e-40d5-b3b6-e4b2d158c1ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:22.825086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:22.826022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:22.826860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:22.826860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:22.826860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:22.830901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 246/494] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:22.832153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe598b-da2e-4d85-a587-f6ebdf2cc31f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.832439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe598b-da2e-4d85-a587-f6ebdf2cc31f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.832709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe598b-da2e-4d85-a587-f6ebdf2cc31f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.834168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 247/495] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:22.834754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe598b-da2e-4d85-a587-f6ebdf2cc31f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.892573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.914537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c67b8-0f74-459d-8cde-2afcd9d14f2d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.938282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:22.944768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe598b-da2e-4d85-a587-f6ebdf2cc31f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:22.948317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:22.952542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:22.982013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:22.982663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:22.982939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:22.983203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:22.994921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 249/496] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:22.994921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:22.999334 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-13decad8-df06-4545-adc0-f46444c8be0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:23.004524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:23.005449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:23.006325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 248/497] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:23.006997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.006997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.007941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.026029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:23.027197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:23.099641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dfc3c-6ebd-4aeb-a260-537b375d440f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.107845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 249/498] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:23.107845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.143740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.144645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.144921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.145191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.145828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:23.146128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.147936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.150192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.165117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 250/499] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:23.165117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.205602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.215072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d913955-117e-4c9a-8636-01efb708cb6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.250129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.250736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.251006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.251278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.264263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cedcdc-bff5-4fc3-b7ba-139c6944718d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.301896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.302495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.302799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.303041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.303697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:23.304030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.305383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.313705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 251/500] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:23.316916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.324825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.326281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.368760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.425269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa5898-06d0-4cf4-80e2-e265751156a4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.425906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 250/501] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:23.426621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.431042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.462200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.462746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.463024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.463292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.474131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:23.479197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:23.480277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 252/502] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:23.485996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.533163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b540-8ef6-47c4-890d-16e78d7c7d6f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.566591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.567209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.567486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.567785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.575040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.576533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.585700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbef98-1287-4800-9484-b8ea6d37f85e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-92205e60-eb4a-4f48-88d4-ab9b46bb238a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:23.589004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:23.598372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 251/503] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3b1d-c0d4-4a43-a1ed-8b3c45c8e7f5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.599297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3b1d-c0d4-4a43-a1ed-8b3c45c8e7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.600019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3b1d-c0d4-4a43-a1ed-8b3c45c8e7f5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.600019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 252/504] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:23.600019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a3b1d-c0d4-4a43-a1ed-8b3c45c8e7f5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.600019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.625099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.625844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.626128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.626401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.637149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.638602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:23.652874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:23.654382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:23.655252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:23.655252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:23.655252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:23.655252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 253/505] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:23.655252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.673091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.701216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a3b1d-c0d4-4a43-a1ed-8b3c45c8e7f5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.734357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.734921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.735231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.735528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.745857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.747313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.752180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8038e6-883e-450c-8713-ba33a207fe38 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:23.760895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:23.762555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:23.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:23.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:23.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:23.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 253/506] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:23.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.789464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.790082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.790359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.790640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.791291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:23.791592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.793043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2ebc-2330-425b-96d3-541bc330be53 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.800608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 254/507] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:23.801739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.802020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.802300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.817116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.818927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:23.820213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:23.863176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0b8f6-8d83-4037-a1e3-6dcdd210befb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:23.884656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 255/508] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:23.885478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.895285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:23.904048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:23.904660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:23.904934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:23.905202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:23.908874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:23.908874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:23.908874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:23.943801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 254/509] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:23.944477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:23.985194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01568a5-5b41-438c-88a4-03c355ef4fb7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.017800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.018402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.018682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.018961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.019599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.019933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.021299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.032256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 256/510] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:24.032890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.045964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59099-7438-4381-8b55-b0d0b57539d3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.059261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.079213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.079832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.080109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.080375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.081002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.081296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.082660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.094950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 255/511] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:24.095713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.098120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.131109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a192691-07b2-4f35-adbe-033794d28904 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.163516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.164157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.164434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.164703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.175320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.176767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:24.191066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:24.192023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:24.192881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:24.192881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.192881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:24.192881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 257/512] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:24.193097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.200473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb2472-b440-44b4-af5e-ac95d2c812aa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.233948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.234560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.234829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.235095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.245285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.246739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:24.260148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:24.261131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:24.261986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:24.261986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.261986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:24.261986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 256/513] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:24.261986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.286611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.291622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc8d78-f9d6-44ec-8997-8cadbf90474c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.311599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.325070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.325657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.325931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.326199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.362092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67124b91-ad48-4db4-aa10-67e77cd6b782 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.391405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.392929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.395043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.395648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.395949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.396214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.403897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.405311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:24.415876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:24.416766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 257/514] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:24.417745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.492630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 258/515] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:24.493832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.499810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.514879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297335d-9719-4485-92c6-9b501acbc2c7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.515037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.547780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.548419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.548741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.549018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.555516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.558125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:24.569709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:24.570707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:24.571614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 258/516] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:24.572451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.647115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7857-52e5-49a2-9b0e-3f85433a1ca7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.680680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57cb63-0183-4c00-8ec3-440ac5260239 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.681384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.681977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.682252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.682522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.683151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.683446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.684853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.697346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 259/517] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:24.698184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.719619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.720262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.720535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.720802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.721435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.721732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.723120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.734805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 259/518] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:24.735457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.742343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.781853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.799932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36a11-c184-4ffa-85b3-33c2c26b41b1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.834015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.834613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.834891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.835161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.835913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.836214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.837630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.839093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74341e7-8ad6-47b1-a4a2-b9401ecece5a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.854842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 260/519] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:24.855632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.877726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.878342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.878618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.878911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.888913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.890355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:24.904663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:24.905704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:24.906554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:24.906554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:24.906554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:24.906554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 260/520] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:24.906554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:24.955011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742d1e3-ee9d-4ad7-8a93-4fe5b34d13bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:24.959753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:24.988106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:24.988700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:24.988980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:24.989251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:24.989933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:24.990236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:24.991644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.002396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.003108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5fe28-f6f5-4977-9230-5a49c1e51cf7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.009458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 261/521] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:25.014761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.037196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.037749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.038026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.038291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.047024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.048749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:25.060975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:25.062018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:25.063071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:25.066938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 261/522] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:25.067809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.110511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039d93a-f0c7-4192-8ac6-e0f0ac3bde67 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.142641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.143239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.143517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.143811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.144436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.145862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.165256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc46337-5bea-4efd-a960-d3d8ead674f1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.181154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.197855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.198447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.199213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.199483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.223469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 262/523] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:25 2022] POST /identity/v3/users => generated 247 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:25.224383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.274230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.275734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.298555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.349315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df73da-b4a8-463d-b374-0b0ccc874b3f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.387628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.388371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.388886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.388991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 262/524] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:25.389069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.389507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.389633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:25.389916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.391227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.404036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 263/525] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:25.405450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.407237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.498296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32dba-a0d1-45bd-bb97-a1525deaf71d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.516628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ae59-d04a-4f1a-8d31-a1d973e8fa11 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.533692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.537100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.537707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.537981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.538246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.539019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:25.539318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.540762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.546991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 263/526] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:25.547901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.567128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.568030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.568314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.568589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.569353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:25.569660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.572219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.599673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 264/527] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:25.603294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.665082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.686984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce406b-5827-4274-ae14-5934f4383cc3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.726854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.727507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.727856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.728047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.734601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872d07b-40be-4b0c-8823-9301e078291b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.739596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.741205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:25.758173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:25.759360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:25.760443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:25.760443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:25.760443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:25.763829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 264/528] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:25.763829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.777516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.778378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.778750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.779089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.786587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.788133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:25.801157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:25.802043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:25.802904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:25.807159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 265/529] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:25.808574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:25.846944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.894903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11429ba-3b12-49e9-91ae-3120494ecd7f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.930316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58093f05-681c-45f6-96a7-77a5675ab9c3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:25.935182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:25.941061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.941793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.942171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.942538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.970420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:25.971851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:25.972274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:25.972704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:25.973510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:25.973964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:25.975697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:25.993520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 266/530] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:25.994693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.027876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.029663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.094439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.108065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f154c5-b5a6-43b2-a9dd-70e364326fe8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.146233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.147150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.147680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.148292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.149342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:26.149881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.151786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.155455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 265/531] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:26.156338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.170517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 267/532] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:26.171717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.191360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.269001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78fa-c9a1-430e-8025-ae5a8822bac2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.303873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fa96a-c710-4b05-a0e2-e096cfb47d2a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.308961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.309595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.309875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.310147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.318536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.320184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:26.336312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:26.337174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:26.338030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:26.345233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 266/533] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:26.345233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.355238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.356062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.356431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.356736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.368809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.370861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:26.387743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:26.388712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:26.389568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:26.389568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:26.389568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:26.389568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 268/534] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:26.389783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.393531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.456221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.467189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90bf8-4912-4ef1-b3f6-43071c17c453 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.502450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f129f9-ace3-40d7-bdac-af49628bc016 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.509649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.510251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.510528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.511256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.512014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=a947c2e6ba544767897eac257c1fad7f) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:26.512323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.514127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.527799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 267/535] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/role_assignments?user.id=a947c2e6ba544767897eac257c1fad7f&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:26.527799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.557657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.559179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.559595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.560091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.561023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:26.561528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.563298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.600931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 269/536] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:26.608350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.647350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcfe5c-f26d-4086-9031-4077d2da7199 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.699296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.699968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.700250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.700524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.701212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:26.701515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.702937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.727127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 268/537] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:26.733398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:26.764147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.772841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:26.833746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337dfe-4ea3-4d28-beaa-894ea3557897 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.889985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbc54d-110c-443c-8825-463889a068d3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:26.904526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.905080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.905362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.905638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.906367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=a1e9a4ceebcc44389144522cec3cc678) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:26.906673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.908133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.920755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 270/538] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/role_assignments?user.id=a1e9a4ceebcc44389144522cec3cc678&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:26.925386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:26.926028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:26.926307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:26.926577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:26.934415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:26.935994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:26.948026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:26.949688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:26.950741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:26.953037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 269/539] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:26.958487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:27.113482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a691d-ac4a-489b-80eb-1ad7810cbb98 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:27.141900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:27.160671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:27.160671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.160671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.160671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.161056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:27.161056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:27.161056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b239ecb-6e4e-4e03-84aa-ee7db11988e1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:27.169892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 271/540] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:27.176867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:27.424224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:27.459505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fdf56-f0bb-4ae7-9f9d-583fe5d6eec5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:27.515109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:27.515565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.516070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.516558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.517846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=c3ab7cc322d04f84ab56c54cf8429596) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:27.518184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:27.520753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dff78d-f59f-411d-98c2-d9473063625a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:27.532354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 270/541] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:27 2022] GET /identity/v3/role_assignments?user.id=c3ab7cc322d04f84ab56c54cf8429596&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:27.648671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dfef9-b167-40d3-afa4-cfec1fc37fc1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.649328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dfef9-b167-40d3-afa4-cfec1fc37fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.649801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dfef9-b167-40d3-afa4-cfec1fc37fc1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.652250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 272/542] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:27.658067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.658556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.659039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.661273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922c8b-8a15-4ee1-8c0e-3e98c2ce318b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.661551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922c8b-8a15-4ee1-8c0e-3e98c2ce318b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.661822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922c8b-8a15-4ee1-8c0e-3e98c2ce318b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.666215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 273/543] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:27.671376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:27.671661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:27.671963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:27.701171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:27.702397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:27.712897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:27.715209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:27.793770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 271/544] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:27.799829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 274/545] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:27.816450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:27.821829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:27.938170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:27.993204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153ad04-0c69-47aa-bb59-18242f32b596 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.004687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f0c56-1e1e-4ae3-8b52-297ccdc37522 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.036894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:28.059664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.060315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.060642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.060917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.071599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.072623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.073220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.073798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.073965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:28.078749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:28.079654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:28.080499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:28.084005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 272/546] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:28.086131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.106471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:28.112808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:28.113695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:28.114583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:28.116410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 275/547] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:28.120606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.272696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b8594-419d-45e8-aecd-7156733e01f7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.315063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0010a-6f42-424b-900e-aafef053c360 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.331339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:28.341881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.353511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.364208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 273/548] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:28.374493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.375096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.375370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.375635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.376409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:28.376833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.378256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.381240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.405890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 276/549] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:28.409911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.485206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:28.581188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5e3-8e0b-470d-9292-78fdfc4eff8e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.581857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeeec49-3910-4a64-8d49-d72422bf49f3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.632716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.633581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.634154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.634684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.647469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.649465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:28.665114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:28.666094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:28.667019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:28.667019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:28.667019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:28.669296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 277/550] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:28.673653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.674662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.675877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.676682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.685858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.704412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.707088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:28.725979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:28.727076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:28.728126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:28.728126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:28.728126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:28.730441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 274/551] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:28.740761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.811789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:28.861960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:28.875423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3d68-7962-4866-983b-72c9fef3284e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.889723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb45a-ae2f-45f8-88ad-0a5294071a26 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:28.911700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.912601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.913055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.913499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.914394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:28.914868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.916472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622dc38-c421-4a43-85bc-66c0ec412192 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.924536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:28.925144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.925425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.925706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.926340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:28.926644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:28.928032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09df05-46e1-4a75-9eb9-d99e2fcba6dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:28.931045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 278/552] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:28.932533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563105-776a-4ebc-b092-48807e5cc730 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.933579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563105-776a-4ebc-b092-48807e5cc730 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.934032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563105-776a-4ebc-b092-48807e5cc730 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.939787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 279/553] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:28.940850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9563105-776a-4ebc-b092-48807e5cc730 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:28.946770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 275/554] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:28.952363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:28.952633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:28.952913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:28.972042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:28.973233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:29.044791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 276/555] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:29.045971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.065075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9563105-776a-4ebc-b092-48807e5cc730 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.079997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.114815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.115455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.115912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.116164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.124446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.125852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:29.137931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:29.139036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:29.140011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:29.143931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 280/556] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:29.144755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.154543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b298-40d5-4175-a704-e21b9a287bfa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.161438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.192243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.192867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.193174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.193444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.201082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.202506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:29.213810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:29.214943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 277/557] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:29.216434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.263461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca37163-60f2-467c-87ed-c4826cb5c153 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.304162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.304786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.305053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.305347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.315234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:29.315692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e84b9ae2-6afd-4dd7-8ba9-454cede4a53b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:29.320238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:29.321245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:29.322130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 281/558] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:29.322898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8be1-29a0-489e-8af9-1ff0719ab9c6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.322898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8be1-29a0-489e-8af9-1ff0719ab9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.324258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f8be1-29a0-489e-8af9-1ff0719ab9c6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.327809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 282/559] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:29.327809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f8be1-29a0-489e-8af9-1ff0719ab9c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.331073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda0c30-1599-4405-8408-45d7c584a80c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.366445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.366973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.367248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.367550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.368368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:29.368666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.370733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.393231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 278/560] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:29.393231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.403426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.435023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f8be1-29a0-489e-8af9-1ff0719ab9c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.472329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.473416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.473762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.474038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.474755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:29.475057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.476715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e11bd4-7565-4c64-8a10-dc2837d2a0c8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.496017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 283/561] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:29.497207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.497495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.497768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.504676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7407cb-2bfa-4cf0-9a86-0692ce258a02 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.515169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:29.516396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:29.543028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.543813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.544419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.544697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.549160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:29.549160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.549160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.564471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 279/562] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:29.564471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.584100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.589369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 284/563] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:29.590122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.653534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.689541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604cba2-0439-437e-8d66-a8e2caf56c9b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.696016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4b39c-0c66-43a1-b681-4f5348a68ce8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.724696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.725315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.725601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.725870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.731682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.732309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.732589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.732861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.736988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.738432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:29.752322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:29.753309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:29.754192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:29.754192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:29.754192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:29.757989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 280/564] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:29.758573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.806135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.807662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.827093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.864878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a2cc-a6f9-4222-9de2-3be2a1618ef7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:29.873060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:29.898127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:29.898746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:29.899026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:29.899295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:29.910150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:29.911530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:29.915409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 285/565] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:29.916457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:29.925239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:29.926190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:29.927081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:29.927081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:29.927081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:29.928013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 281/566] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:29.928690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.016310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298205e-1d40-4b32-9b8d-1b950ccc9981 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.025657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13cd66-3ac4-4cb5-a7b4-e976e23b6b0f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.042630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.049415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.050034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.050314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.050585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.058695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.059275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.059548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.059840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.060466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.060578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:30.060762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.062140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6eaa74fa-c7e2-40d3-b4d2-5abf80b8f00e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:30.065028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.066157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:30.067066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:30.068395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 286/567] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.069658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7cd18-cfbc-49a9-befe-c6b5e9036ec6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.069990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7cd18-cfbc-49a9-befe-c6b5e9036ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.070407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7cd18-cfbc-49a9-befe-c6b5e9036ec6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.071854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 287/568] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:30.072550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7cd18-cfbc-49a9-befe-c6b5e9036ec6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.074107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 282/569] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.074900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.098988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.175329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fef3e-3a9f-4acd-8cd5-48dc445f3c4a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.177101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7cd18-cfbc-49a9-befe-c6b5e9036ec6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.208841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.209171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.209439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.209719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.209821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.209988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.210078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.210285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.210618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.210930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.212335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.220608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.222031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.223931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 283/570] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.224711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4233dc79-ca70-458c-bd24-bbfdb9d47255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:30.236562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.237751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:30.238665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:30.238665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:30.238665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:30.238665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 288/571] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.239080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.239360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.239635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.257231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:30.258418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:30.267395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.324789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 289/572] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:30.325599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.327350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddceab0-16d3-4f8f-a8c4-f9056c057616 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.327961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.362714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.363310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.363581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.363880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.364523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.364818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.366186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.374476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 284/573] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.375248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.424595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9537d-e049-4e6c-8e41-2389ba36f430 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.458172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.458788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.459067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.459345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.465886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.467614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.471162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.474010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3eaf6-d2a9-45ad-add3-d77d898ff1b7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:30.478751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.484244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 290/574] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.485132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.509987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.510640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.510919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.511190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.519174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.520644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:30.531441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.532565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 285/575] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.533565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.544677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.581256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b3514-dd93-4484-a111-d97b35a5b110 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.616264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.618124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.618124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.618124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.618124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.618441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.619733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.635445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 291/576] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.635445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.641981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5438f1-3165-42f8-8fc5-3195844d3d76 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.674923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.675539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.675844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.676116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.685953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:30.691061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:30.692950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 286/577] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.693702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.714547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.737560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87843b-5093-4891-9f96-b55236bcaf07 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.759553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.770551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.771164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.771436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.771705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.782355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.783803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.793181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcfddf-70f7-46fa-a677-fbc8fade8500 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:30.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:30.798860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:30.799750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:30.799750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:30.799750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:30.799750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 292/578] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:30.800326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.827189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.827802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.828083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.828356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.829032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.829333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.830751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.848492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 287/579] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.849272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.900271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9ff9c-8b6f-4b4c-b92c-b1261fc4dd01 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.930548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.933342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.933934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.934214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.934484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.935198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:30.935578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.937011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:30.949173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ecd4f-78c3-45f6-8c39-87b11b5b2e1d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:30.953495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 293/580] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:30.954288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:30.984226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:30.984770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:30.985048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:30.985316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:30.985465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:30.992056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:30.993503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:31.004834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:31.005883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 288/581] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:31.006770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.053497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5828a3-5a76-42bc-9f91-f51b85b8c7fc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.087469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.088101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.088387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.089172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.089836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.090136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.091480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.099529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 294/582] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.100321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.110408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f23018-9cfd-4d85-bf00-273b4dbe2f8a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.143577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.144117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.144394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.144665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.145296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.145595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.146946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.157236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.160453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 289/583] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.161286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.192681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.199197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330bde9-1581-4e1b-a30d-1dbbe46d120c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.233149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.233793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.234068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.234391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.249117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=d622a3b3cc2a4e469033cb2d5c66d2ed, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.250617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.257647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445516c-5379-413c-9290-14bf8e3dc8c6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.290796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.291406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.291687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.292006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.301718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.303252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:31.317176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:31.318301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:31.319331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:31.319331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:31.319331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:31.319331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 290/584] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:31.319331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.322619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 295/585] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:31 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:31.323934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.381544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.435838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cedfc4-a3a2-407e-9845-452b6ce0c8c9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.438425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36002545-058a-417c-91e8-1b4a5c029e82 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.475608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.476269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.476544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.476812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.482015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.482810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.483094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.483420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.484148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.484504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.485907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.491088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.492663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.495883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.500005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 291/586] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.501089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:31.508174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:31.511417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:31.512568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:31.512568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:31.512568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:31.512568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 296/587] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:31.512568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.623384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429a983-9317-4759-b969-7febc9d84b84 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.623911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3af57-e03a-464d-9ccb-77f55793268b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.638498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.659165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.659793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.660065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.660341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.660963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.661263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.661453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.662084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.662372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.662613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.662720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.671140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.672332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.676403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 297/588] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.680566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:31.685135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:31.686254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:31.687510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:31.693211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 292/589] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:31.693211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.742308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.787176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb170bd-c706-4d42-98d9-2387360533e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.795228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542819-67ea-4641-8a9a-05a9ada58515 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.823554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.824197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.824474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.824743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.825462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d0dea5e1e47b48b0b7623c6b7100279e) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.825773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.827181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.843796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 298/590] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/role_assignments?user.id=d0dea5e1e47b48b0b7623c6b7100279e&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.843796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.848132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:31.848837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:31.849186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:31.849548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:31.850272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:31.850644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:31.852296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:31.867217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 293/591] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:31.868295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:31.910936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:31.962209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249039a-94ec-4e73-a718-0c115659f207 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:31.985244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.001760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.002710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.003283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.003704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.006471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa951-a17b-4fd1-8a2d-d2aef8b84863 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.019177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.019177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:32.039882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:32.040841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:32.042331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 299/592] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:32.042826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.050791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.051399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.051671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.051968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.052700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:32.052998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.054528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.074561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 294/593] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:32.075464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.152022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1f9d2-6cf6-4a11-aa7c-e3c2077bb48a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.185552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.186174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.186454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.186724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.187289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.194352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.195115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babd4f1-2955-4114-8f52-dd40aaf31c2a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.196196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:32.208754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:32.209789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 300/594] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:32.210775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.239675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.240376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.240957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.241241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.242599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:32.242920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.245104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.255597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.262846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 295/595] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:32.264242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.326641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a79a7b-2ea9-4686-9db9-9803c7c3d994 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.371541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.372375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.372729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.373064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.384711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.386302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.401881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edc396-5920-48fb-956c-fa84f57b8414 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:32.407102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:32.408291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:32.409575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:32.409575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:32.409575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:32.409575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 301/596] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:32.409575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.448872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.449572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.449982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.450320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.451111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:32.451483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.453198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.472855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 296/597] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:32.473801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.485287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.518394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.527059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1203e-5aa3-488b-8f29-6f4719e1395c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.562652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.563391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.563743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.564114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/a947c2e6ba544767897eac257c1fad7f/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.579583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=a947c2e6ba544767897eac257c1fad7f, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.581223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.585376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b342d9-3ecd-4d63-bf37-dba6d0ae31e5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.622753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.623593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.624067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.624541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.625391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:32.625878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.627421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.636586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 302/598] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:32 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/a947c2e6ba544767897eac257c1fad7f/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:32.637309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.641729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 297/599] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:32.646818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.746069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f058da3-b1ec-4149-a9c5-dc7423309bf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.811909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.812468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.812740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.813008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.826380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.829781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.832024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:32.851740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:32.852784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:32.853679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:32.853679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:32.853679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:32.855045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:32.857110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 303/600] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:32.859106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.872586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6aafd-fe31-4fa9-9608-8f8cf550d87f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:32.909900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:32.910886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:32.911367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:32.911868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:32.920496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:32.922204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:32.934522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:32.935456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:32.936421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:32.938812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 298/601] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:32.944746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:32.968933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9bc47-573b-41eb-ace7-bca369fee6bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.006108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.006682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.006963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.007231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.007902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:33.008521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.009621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.022498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 304/602] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:33.027842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.071785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b68226-18ef-40ed-ba74-257b3584d99f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.104929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:33.114391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.115425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.116051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.116527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.117413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:33.117960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.119855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.135006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 299/603] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:33.141034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.176138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:33.256203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71039732-61d7-4360-81ad-cb14475c4066 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.280365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7bc9c-d0f1-4b5a-b79b-2f067802ca27 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.313449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.314118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.314396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.314662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/a1e9a4ceebcc44389144522cec3cc678/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.332619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.334697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.334974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.335236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.347286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=a1e9a4ceebcc44389144522cec3cc678, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.348766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f1149-eaee-47f7-9299-267377e18b13 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.351451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.352914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:33.365989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:33.370641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:33.371546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 305/604] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:33.373687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.459574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 300/605] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:33 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/a1e9a4ceebcc44389144522cec3cc678/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:33.465411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8816b-59f2-4685-ac9b-dbca33d89915 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.466072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8816b-59f2-4685-ac9b-dbca33d89915 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.466599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8816b-59f2-4685-ac9b-dbca33d89915 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.470412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 301/606] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:33.476147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.476782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.477261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.511609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd68650-0530-4e03-b3de-dd36faa9bf9b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.522537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:33.527500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:33.548180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:33.585025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:33.614877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.615696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.616095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.616485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.617289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:33.617692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.619216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.643652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 306/607] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:33.650317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.664982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 302/608] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 11 10:43:33.693537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.794502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10faaf-0c10-4c99-8aec-822b94787b58 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.854441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0ffff-03cb-4516-b500-1b265dc182ee None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:33.857809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.858469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.858796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.859075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/c3ab7cc322d04f84ab56c54cf8429596/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.888213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=c3ab7cc322d04f84ab56c54cf8429596, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:33.889665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d068fd-b81d-4b62-9a9b-40ed741a6f50 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:33.909943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:33.910701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:33.910931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:33.911204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:33.922227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:33.927170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:33.928146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:33.928982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 303/609] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:33.935004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:33.966368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 307/610] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:33 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/c3ab7cc322d04f84ab56c54cf8429596/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:33.989854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.075449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fb33b-c7c9-43a0-a2bf-7117d6b48698 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.095074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebc792-a2c5-4692-8e35-14a2ed3de78e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.095612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebc792-a2c5-4692-8e35-14a2ed3de78e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.095699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebc792-a2c5-4692-8e35-14a2ed3de78e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.097284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 308/611] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:34.102331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.102610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.102878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.132779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:34.133693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.134578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.135053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.135708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.136595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:34.137079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.138745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:34.141867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:34.158126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 304/612] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:34.168256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.183672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.255928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 309/613] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 10:43:34.288276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.301085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.311616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca58762-a3af-41e7-a708-ab3ea2f04730 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.348527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.349827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.350202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.350572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.362029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.363644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:34.378245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:34.379455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:34.380446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:34.380446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:34.380446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:34.381155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 305/614] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:34.398337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.411671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed68c52-42b6-46a9-a3b9-f71af59fdf39 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.487708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.488369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.488653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.488931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.524687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:34.529921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:34.530886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:34.531738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:34.540206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 310/615] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:34.555512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.559362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf725c-0ed0-4dd1-a54c-99c0cd00879e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.598287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.599034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.599388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.599736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.600521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:34.600901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.607235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:34.607428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.622959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 306/616] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:34.624556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.640138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.737796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faacdd-854a-4c0c-8289-baa391646b27 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.774759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.775504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.775928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.776276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.788099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.790024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:34.804601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:34.805606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:34.806563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:34.811331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 307/617] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:34.812579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.827723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417f7e-00d2-43bd-840d-c85dec8ed82f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.892272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:34.905000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.905961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.906439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.907068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.907951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:34.908468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.910021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5427e1-e24b-454f-a6ea-8000eaf91301 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:34.928361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cd9ca-1e4a-455b-8009-d9f30aa6bc67 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:34.930964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 311/618] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:34.939554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57d163-695b-4c8e-8bd5-4a49fbe36849 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.939859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57d163-695b-4c8e-8bd5-4a49fbe36849 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.940135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57d163-695b-4c8e-8bd5-4a49fbe36849 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.941687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 312/619] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:34.946017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57d163-695b-4c8e-8bd5-4a49fbe36849 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:34.976059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc154908-fa58-47af-9400-830592e232dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:34.976730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154908-fa58-47af-9400-830592e232dd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:34.977015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154908-fa58-47af-9400-830592e232dd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:34.977336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154908-fa58-47af-9400-830592e232dd None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:34.978123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc154908-fa58-47af-9400-830592e232dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:34.978430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc154908-fa58-47af-9400-830592e232dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:34.980519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc154908-fa58-47af-9400-830592e232dd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.003707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 308/620] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:35.004955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.005246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.005524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.023256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:35.024481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:35.031862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.058130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57d163-695b-4c8e-8bd5-4a49fbe36849 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.095976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 309/621] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:35.097132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.105932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.106578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.106848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.107121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.126449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.128027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:35.156166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:35.157195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:35.158105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:35.158105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.158105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:35.161889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 313/622] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:35.162715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.167091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.209887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4675f2-5fd8-4c58-bfde-b61bfa1bdc0e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.247026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.247721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.247930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.248236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.303497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ea30c-924e-4e38-a12a-a4def27e38bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.326485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.328714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.332647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.342560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.343258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.343673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.343983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.354982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:35.360247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:35.361418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 314/623] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:35.362681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.392902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.455581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 310/624] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:35.456490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.514405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf8a89-2dde-4918-91fb-e2e4ee7c7224 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.549723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.550332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.550711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.550991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.551636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:35.556011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.556011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.570492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 315/625] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:35.572049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.574161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ab7f4-d8b2-477d-8a52-c8a66d6bb4cf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.611798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.611798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.611798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.611798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.612255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:35.612255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.612678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.625954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.626165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 311/626] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:35.627072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.642894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.682662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c04a-217e-4344-80c9-2daecb06d4de None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.721024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.722032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.722514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.722946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.730751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9571c-a73a-4e5a-991e-75036d5d7eab None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.734958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.736771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:35.756659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:35.759634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:35.761311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:35.761311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.761311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:35.763057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 316/627] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:35.764104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.781599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.781599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.781599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.781984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.790040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.792067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:35.803587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:35.804648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 312/628] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:35.805690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.880309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.884318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:35.888560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c2d0a-74ea-410e-ac3e-bbac88c19dea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.908474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed07b9-7efe-484e-93ed-b2baca0f2815 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:35.922909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.923528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.923827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.924100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.934737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.936189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.943561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:35.944178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.944452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.944723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.945359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:35.945659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:35.946997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e62b704d-b1d9-4c54-b903-161aa643c6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:35.951548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:35.952486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 317/629] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ff8e-72e7-4178-ace6-66aff02e877a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ff8e-72e7-4178-ace6-66aff02e877a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866ff8e-72e7-4178-ace6-66aff02e877a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:35.958278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 318/630] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:35.960471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866ff8e-72e7-4178-ace6-66aff02e877a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:35.969938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 313/631] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:35.969938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.064852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866ff8e-72e7-4178-ace6-66aff02e877a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.069663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21488-89f0-4dfe-a525-79f090f13358 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.100412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.101012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.101286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.101809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.101915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.102377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.102561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.104033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853419bc-3487-4f92-8fb6-4ba974a97bc5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.108917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.109525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.109826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.110118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.110712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.111002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.112401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.123811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 319/632] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.125035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.125319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.125592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.126158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 314/633] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.127160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.148393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:36.149513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:36.153275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.221947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 320/634] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:36.222774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.248326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5c3c8-ab53-4a9e-a5e1-6d436ddfde16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.284722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.285343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.285617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.285882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.292637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.294726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:36.306271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:36.307746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 315/635] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:36.308728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.330767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298cb21-e592-4937-a904-057d5e9b709a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.364117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.367582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.368234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.368513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.368781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.377339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.378757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.381261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:36.390507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:36.391650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 10:43:36.392865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:36.396295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 321/636] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:36.397065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.420236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe1c33-9625-4edb-a5fe-0fe09b4115f1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.453888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.454494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.454776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.455042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.465194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.466634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:36.480835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:36.481959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:36.483092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:36.483092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:36.483092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:36.483092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 316/637] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:36.483092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.499033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eb16-2b5a-43a5-862f-7b897f902987 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.532123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.532755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.533027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.533292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.543260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:36.548240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:36.549933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:36.550966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:36.554346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 322/638] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:36.555076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.580062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3fbb0-47c1-4cd1-9e2b-489254c6251b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.582332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.618826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.622008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.622613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.622882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.623145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.623886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.624215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.625637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d991bd8-9bfc-47d9-bde5-231f21063d72 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.638512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 317/639] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.639700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7539-de5d-4ded-b68a-c54bc45acefb None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.640007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7539-de5d-4ded-b68a-c54bc45acefb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.640276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f7539-de5d-4ded-b68a-c54bc45acefb None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.641543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 318/640] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:36.642221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f7539-de5d-4ded-b68a-c54bc45acefb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.663667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e773c1d-ee85-41a7-b6d5-99712f283ec7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.696120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.696716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.696984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.697255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.698464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.698768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.700195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.717512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 323/641] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.718303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.743340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f7539-de5d-4ded-b68a-c54bc45acefb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.778053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.778798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.779162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.779439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.780105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.780406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.781769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb372db0-2b8c-4753-a97e-4b1912440ef6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.792610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 319/642] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.793747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.794033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.794302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.811129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:36.812333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:36.813404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fee1a-6c82-485e-a47b-e5a4a4fde543 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.829725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.846514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.847101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.847375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.847639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.848308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.848606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.849940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.857574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 324/643] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.858340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.878842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 320/644] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:36.879663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:36.955298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129efb-dbfc-423c-bdc2-6c77ab2c8e29 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.963774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:36.974472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffd2c3-dfb2-49e4-9f25-4daf0ea21824 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:36.989180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:36.989752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:36.990027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:36.990296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:36.991010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:36.991314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:36.992745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:36.998897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 325/645] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:36.999484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.016968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.017699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.018249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.046255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.053956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.093634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.095120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.119683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d466d-1a4a-4514-886d-ed6ede36b1f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.162470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.163125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.163537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.163654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.170686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.172181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:37.184914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:37.185932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 326/646] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:37.187243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.210466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 321/647] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:37.216191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.277077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.301715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b28-eced-40a9-877d-d68e6bb2f31e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.306183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.321710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fb6b-77a1-4cd9-96d7-001b30a41cc3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.335694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.337219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.337219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.337927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.348633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.350068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.358759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.359379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.359652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.359948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:37.367233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:37.368241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:37.369183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:37.369183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:37.369183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:37.369183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 327/648] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:37.369183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.375554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:37.380266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:37.381440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 322/649] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:37.382458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.477218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7be8ca-7b51-4827-9163-79f5defe960d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.487799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13711167-1a89-41ef-8e07-b20f2a5ea427 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.501827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.512186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.513914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.513914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.513914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.514289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:37.514445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.516033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.524301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.524929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.525211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.525483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.528566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 328/650] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:37.529511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.537468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.539153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.549100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:37.556045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:37.557580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:37.558505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:37.558505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:37.558505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:37.558505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 323/651] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:37.558505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.653537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096957aa-0f21-4f58-b775-00b64c0a09c3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.670257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ad80-6ccf-449f-a646-0b63f7878f14 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.688763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.689362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.689633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.689898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.690525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:37.691234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.692646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.704638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.709141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 329/652] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:37.709257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.709257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.709257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.709515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.709608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:37.709608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.709608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.717950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 324/653] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:37.718682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.748427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.794239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:37.817504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2b39f-f90b-4567-b2da-f884aec57d35 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.823079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675667c2-e8f2-4f92-bfe1-e0816540c2a9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:37.852593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.853243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.853687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.853907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d0dea5e1e47b48b0b7623c6b7100279e/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.856923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:37.857588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:37.857930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:37.858201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:37.858832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:37.859130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.860513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.870769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=d0dea5e1e47b48b0b7623c6b7100279e, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:37.872745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:37.873159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 325/654] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:37.873941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.937368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 330/655] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:37 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d0dea5e1e47b48b0b7623c6b7100279e/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:37.938194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:37.991802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.011358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c0659-4359-47af-867f-f95b5c564334 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.054094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.054732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.055006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.055277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.064360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.065840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.077148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502bf7c-100e-4ad1-a19d-efda08179c46 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:38.078621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:38.079589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:43:38.080595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:38.084946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 326/656] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:38.085914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.118150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.131841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.132477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.132753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.133021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.145858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.147284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:38.163156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:38.165077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:38.166024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:38.166024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:38.166024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:38.166024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 331/657] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:38.166024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.219410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977f1b-17c3-49c9-8a60-bfcc83fd406b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.265263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.266134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.266474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.266994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.276850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5d964-337b-4084-99f7-7ee7a4f3d039 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.277087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.278588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:38.299195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:38.300949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 327/658] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:38.302583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.317533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.318279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.318668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.319045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.319824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.319988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:38.320755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.322983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.342966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 332/659] 10.222.0.48 () {56 vars in 1137 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:38.348235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.387814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.435613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd7cf-d701-4a02-ac07-1a734e9ac467 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.459670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ed1e-343a-494d-b792-e818d80c1207 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.479731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.480627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.481057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.481487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.482774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:38.483347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.485257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.498059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.498905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.499285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.499657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.500469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:38.500888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.502233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 328/660] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:38.503283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.506929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.524727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 333/661] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:38.524727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.593151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.635937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31329996-fc80-44c1-86d7-359cbd479735 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.644843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.653827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c9508-f480-480f-b0fa-9a12c777d428 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.674570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.675178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.675455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.675723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.702466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.703503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.704332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.704885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.713542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.715298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:38.728193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:38.730636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 334/662] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:38.733193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.760667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.762117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.830216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:38.867286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e4f2a-9105-4a36-8b81-2b3e60f8eca3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:38.868393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 329/663] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:38.872753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.908913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:38.909668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:38.910048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:38.910423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:38.911258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=a947c2e6ba544767897eac257c1fad7f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:38.911705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:38.913298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:38.920412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 335/664] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/role_assignments?user.id=a947c2e6ba544767897eac257c1fad7f&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:38.921384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:38.946796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.066918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4bd33-7b07-4c15-b933-16f841bff18e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.067598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375dba7-d7c8-4879-be1a-916ad7641ec7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.116609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.117147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.117420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.117687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.118412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:39.118710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.120695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059d175-a827-41df-a7ea-7efac057a322 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.138169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.138814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.139108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.139368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.147240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 330/665] 10.222.0.48 () {56 vars in 1139 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:39.149451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6771-2070-493b-bfaf-b19d4c58aa5d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.149965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6771-2070-493b-bfaf-b19d4c58aa5d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.150876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6771-2070-493b-bfaf-b19d4c58aa5d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.156697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.157256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 331/666] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:39.158120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.158235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c6771-2070-493b-bfaf-b19d4c58aa5d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-838c94b7-bcdb-4ce7-a44a-addf387e42a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:39.173377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:39.174711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:39.176594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:39.176594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:39.176594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:39.176594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 336/667] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:39.176594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.177607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.177607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.196929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:39.198975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:39.201476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.277646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 337/668] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:39.278481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.284382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.326825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c6771-2070-493b-bfaf-b19d4c58aa5d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.368338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.369775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.370278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.370752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.392820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddb726-fe5d-40f3-b68b-32e6d2a91a4a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.430007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.431060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.432107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.432586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.433454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:39.433961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.435639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.448277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 338/669] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:39.449567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.460572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.462448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.491358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.536190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.573497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 332/670] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:39.574558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.579993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a5ed4-34c4-44bf-be63-1f8a306380c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.621548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.622146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.622424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.622694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.634442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:39.640254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:39.641403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:39.642242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:39.646921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 339/671] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:39.647841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.690296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7834696-9b3d-432d-beef-85cd1d23e8ce None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.717813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.730002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.730780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.733556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.733556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.740290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.741941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:39.754735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:39.756088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 333/672] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:39.758214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.771443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc45ae-b458-4283-a751-c64976df7731 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.796087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:39.809570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.813716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.813716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.813716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.824412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.826162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:39.843383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:39.844618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:39.845598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:39.845598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:39.845598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:39.851694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 340/673] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:39.852931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.863513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa8dd-2b83-4851-943d-cd00622f2f45 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:39.905311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:39.905850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:39.906130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:39.906403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:39.907083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:39.907387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:39.908908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:39.923519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 334/674] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:39.923519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:39.976098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.029895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402ef64-fdf7-49f3-ba34-e73903e773f2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.065178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.065784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.066062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.066409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.066967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.067515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.068647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.082382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 335/675] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.083426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.093091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0605-ef69-4c73-9de7-afce10d90aea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.129212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.130211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.130719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.131167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.132048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.132596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.134381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd90682-5fd0-42b0-9ad2-94f4aff37605 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.147702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 341/676] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.149245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d25a1-41b2-4bf1-8d6b-101928b2eb90 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.149525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d25a1-41b2-4bf1-8d6b-101928b2eb90 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.149797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d25a1-41b2-4bf1-8d6b-101928b2eb90 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.151205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 342/677] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:40.153605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d25a1-41b2-4bf1-8d6b-101928b2eb90 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.190896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865534-3bb3-4cef-8283-cdb36bf831a2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.197388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.201060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.229069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.229699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.229972 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.230241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.241088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.242587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bbd8f33c-7536-49d6-be6c-b41ad6c96bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:40.260902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:40.261944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:40.262846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:40.262846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:40.262846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:40.262846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 336/678] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:40.263641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.263949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.264220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.275420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d25a1-41b2-4bf1-8d6b-101928b2eb90 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.282194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:40.283385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:40.314044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.314940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.315220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.315490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.322307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.323723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:40.335264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:40.336234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 343/679] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:40.342307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.358195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 337/680] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:40.359361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.443489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.450672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.451541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be38c9-faf6-4e9f-b87e-dc2d48e0a360 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.459992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6047c-645f-4d63-876c-97fcda404100 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.491498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.492590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.493111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.493563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.493668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.494250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.494525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.494787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.495173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=a1e9a4ceebcc44389144522cec3cc678) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.495407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.495703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.496018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.497095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.498012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.504064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 344/681] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/role_assignments?user.id=a1e9a4ceebcc44389144522cec3cc678&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.505047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.517542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 338/682] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.518805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.629885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ac08c-f2be-437a-bbbc-a590ec8ec552 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.666065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42e99-0fc8-4d66-aa87-31b7ff76a46a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.669043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.669702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.669977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.670249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.670935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.671237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.672701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.691508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 345/683] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.692699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.714166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.719359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.719736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.720096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.720361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.731798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:40.736889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:40.745871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 339/684] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:40.746953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.753085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:40.812522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862bb507-0cac-4cec-8f10-6a9385375583 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.851788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.852577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.852956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.853408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.859491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd5d43-66e7-4901-ad40-55b76d78ddc1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:40.861464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.863148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:40.874993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:40.876182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:40.877354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:40.879092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 346/685] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:40.884123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.896148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:40.896768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:40.897045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:40.897318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:40.898447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=c3ab7cc322d04f84ab56c54cf8429596) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:40.898933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:40.901030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:40.908971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 340/686] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/role_assignments?user.id=c3ab7cc322d04f84ab56c54cf8429596&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:40.910359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:40.999702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.029171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bc619-532c-44c3-a75e-dd78436b59a0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.039666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.069797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.070576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.070858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.071127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.072797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:41.073119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.074574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.088245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96d42-8466-4c41-bc75-d19edc4111cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.097106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 341/687] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:41.104013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.139791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.140436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.140884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.141186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.141859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:41.142159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.143583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.154242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 347/688] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:41.161560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.261507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14888ec-c1ed-4cba-b37b-f11b286a7173 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.299234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4abac-557a-4f3f-8340-4ba9976439fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.300352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.306776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.308196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.308686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.309640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.342489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.343120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.343398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.343666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.355902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.359427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.360903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-96dd6a88-9864-477d-9056-2b4f0ff1a61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:41.381625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:41.382668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:41.383571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:41.383571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.383571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:41.383571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 348/689] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:41.384333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b64cf-ea7c-4fa2-aead-45a3c45c8d3d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.384649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b64cf-ea7c-4fa2-aead-45a3c45c8d3d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.385185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b64cf-ea7c-4fa2-aead-45a3c45c8d3d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.386949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 349/690] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:41.388301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b64cf-ea7c-4fa2-aead-45a3c45c8d3d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.398516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.401091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63a1b5-6dc0-4161-8877-1c59da52fed8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.518949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 342/691] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:41.520562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.521096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.521568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.528415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b64cf-ea7c-4fa2-aead-45a3c45c8d3d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.543946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:41.545186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:41.573602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.575269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.576256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.576671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.577045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.577915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:41.578333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.580478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.602665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 350/692] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:41.606522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.632275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 343/693] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:41.633257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.674478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.723460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeed45a-e1e8-4cf6-b11b-3ad0968021e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.759500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.760169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.760447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.760720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.765301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8359d3-bbfe-41b2-a853-9e2e8331ff47 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.771113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.772561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:41.787400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:41.788436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:41.789340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:41.789340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.789340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:41.793739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 351/694] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:41.794687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.809772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.810392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.810663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.810940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.818583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.820080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:41.832739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:41.833731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:41.834677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 344/695] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:41.835289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:41.869367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.912912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c9d18-6964-4c9a-8d0d-83d4395d6d94 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.917062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:41.955908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:41.956535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:41.956815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:41.957090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:41.971213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ed2fd-5604-436c-9520-c9377229d871 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:41.972840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:41.975301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:41.994360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:41.995330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:41.996287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:41.996287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:41.996287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:41.997209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 352/696] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:41.997941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.018102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.019602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.020122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.020634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.021542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:42.022585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.024452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.045876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 345/697] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:42.045876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.128933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4962-11bc-4b0a-8bc2-5d6bbf80785a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.141037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.160701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f03d4-81b7-4931-844d-c68cfb29ad66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.169322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.170073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.170695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.170695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.171618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:42.172048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.173566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.194272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 353/698] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:42.195741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.201082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.201931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.202359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.202781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.203636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:42.204185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.205791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.213817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.220702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 346/699] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:42.221918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.304993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ee430-c507-4b19-826f-c8e20671003b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.327380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864cb66-a8c2-4c6e-ba3d-03db9b488b16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.341400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.342427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.342832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.343209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.350344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.351928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:42.364295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:42.365372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:42.366298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:42.368162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.369611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.370107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.370585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.371290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 354/700] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:42.372104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.388129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.448185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.449839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.464031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.490003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835dc5b-187a-4d33-8127-e3b3d17891d6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.526110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.526902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.527188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.527461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.528126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.530436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.561807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 347/701] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:42.562955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.614073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.643857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 355/702] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:43:42 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:42.645168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.687295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b7a0-2805-4a7a-8b18-04a2fc688aa9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.762110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.762728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.763009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.763283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.764006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:42.764312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.766053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc30847-2c24-487b-8753-6c20f5c973f3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.775429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7cf33-76e2-4f54-a763-257cf1d80345 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:42.791745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 348/703] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:42.793092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a541cb-bb3f-4747-99ee-ca6f0d99f07a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.793376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a541cb-bb3f-4747-99ee-ca6f0d99f07a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.793648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a541cb-bb3f-4747-99ee-ca6f0d99f07a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.799242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 349/704] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:42.800638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.800923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.801201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.817562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.820444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:42.821762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:42.822549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:42.823447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:42.823962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:42.824760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:42.840192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:42.842719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:42.859396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:42.860633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:42.861586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:42.861586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:42.861586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:42.864939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 356/705] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:42.871614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.903358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 350/706] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:42.904253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:42.938734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:42.995960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f888b5-610f-4c75-9be5-ee0bf1699990 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.030932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b6099-8c99-4f70-8dfa-f9851ae45e60 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.042104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.043404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.043990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.044415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.045570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.046507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.048716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.057136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 357/707] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.063804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.080454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.082153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.082432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.082758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.083432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.083738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.086963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.112758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 351/708] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.118389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.157446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.222770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26aceb-f26a-4cf1-9e26-5f2000daa4b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.256757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.267711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03fd28-4bdc-4d06-86c1-575d7f8c0dd6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.276276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.276276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.276276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.276276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.276682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.276682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-64cc6097-212a-43f8-a341-3209982b3ded None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:43.285988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:43.287247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.288289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 358/709] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:43.289636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f8ca-5eb5-4cb5-a320-5bc81f59c281 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.289636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f8ca-5eb5-4cb5-a320-5bc81f59c281 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.289636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f8ca-5eb5-4cb5-a320-5bc81f59c281 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.294814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 359/710] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:43.294814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072f8ca-5eb5-4cb5-a320-5bc81f59c281 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.319846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.320839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.321400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.322407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.329778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.331559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-330d12a5-6939-4cdd-ba98-6b7b60f38d88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:43.343642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:43.344614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:43.345538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:43.348943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 352/711] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:43.350428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.350912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.351380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.370820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:43.372411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:43.399279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072f8ca-5eb5-4cb5-a320-5bc81f59c281 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.412332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.435507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.436158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.436619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.436727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.437283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.437664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.438930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.443707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 360/712] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.444516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.457513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 353/713] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:43.458791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.563144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4c35-76a4-4313-aab7-ca3dae691916 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.563846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.584678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514279fb-9b40-4358-89ce-c16e84ac95e3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.601301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.601910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.602190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.602463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.603337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.603535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.605367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.618040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 361/714] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.619491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.629615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.630622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.631106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.631595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.632465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.633204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.634985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.641563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 354/715] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.647937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.659334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.731073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a87db-7e87-4d4b-aa4e-36325a595587 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.754389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53343661-f1b4-498e-90d4-6e2fb3d447e9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.768098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.768735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.769008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.769276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.769860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.771353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:43.780933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:43.782011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:43.788597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 362/716] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:43.802959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:43.804597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.805460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.805962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.806963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.807971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d0dea5e1e47b48b0b7623c6b7100279e) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.808488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.810150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.817335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 355/717] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/role_assignments?user.id=d0dea5e1e47b48b0b7623c6b7100279e&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:43.939902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:43.944673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894cd09-1dc3-4696-b37c-82f7dcbf6a1d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:43.983478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:43.984620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:43.985110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:43.985591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:43.986465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] RBAC: Adding query filter params (name=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:43.986978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:43.988669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8244d-5aea-4e41-8a00-0f38f18f2854 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:43.998570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 363/718] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:44.000291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:44.014481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:44.109722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baac196-a04c-494a-a669-ed275cb08cdb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:44.145659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:44.146602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:44.147104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:44.147596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:44.148436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:44.148944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:44.150553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88b780-92c7-4f02-b309-fa1ca44e650a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:44.160064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 356/719] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:45.031569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0f4-6ab0-4954-b39b-24f269fc4c4b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.032128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0f4-6ab0-4954-b39b-24f269fc4c4b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.032128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bb0f4-6ab0-4954-b39b-24f269fc4c4b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.033678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 364/720] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:45.039030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.039452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.039657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.064735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:45.065929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:45.135220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 357/721] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:45.174830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb0f4-6ab0-4954-b39b-24f269fc4c4b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:45.301060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bb0f4-6ab0-4954-b39b-24f269fc4c4b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:45.348438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:45.349062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.349343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.349618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.362545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e8b9beba-802d-4317-9110-3907ad840e14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:45.367221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:45.368189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:45.370563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 365/722] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:45.374577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:45.377728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:45.581834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87aa52-c47e-43e0-805d-79608778a99d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.582221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87aa52-c47e-43e0-805d-79608778a99d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.582328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87aa52-c47e-43e0-805d-79608778a99d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.583712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 366/723] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:45.602331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.603544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.603544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.622682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fc7aa-8b7f-407f-a4af-cf82ac45ab8a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:45.653017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:45.654978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:45.668953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:45.669557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.669829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.670096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.670755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:45.671056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:45.674541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:45.688107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 358/724] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:45.693907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:45.698685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:45.782567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 367/725] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:45.800652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:45.817165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:45.860339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b5433-3e13-49e9-8404-6d64307f813f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:45.922424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:45.923064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:45.923340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:45.923609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:45.947146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:45.948674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:45.974359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:45.976795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:45.979056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:45.979056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:45.979056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:45.980249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 359/726] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:45.984857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:45.989693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173453d-c6f2-412b-b7a0-cf55862233b9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.057436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.058213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.058396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.058577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.083275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:46.088929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:46.091179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:46.093535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 368/727] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:46.100889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.137816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.150128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cb8c9-cd5d-4301-a30d-170a3c35c585 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.168121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.189735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.190344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.190617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.190884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.191523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:46.191849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.193204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.213154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 360/728] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:46.220823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.292842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8d7f-f113-4b43-b6e9-aa5815e7221f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.338933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.339541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.339841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.340111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.341016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:46.341315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.343605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a9db2-8f38-400e-a6ff-ddaeec41302e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.348101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40185c-1cec-421e-9f28-2aa54c0965b4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.354415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 369/729] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:46.356079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583437-6094-483d-83ec-c1cee4103e53 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.357086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583437-6094-483d-83ec-c1cee4103e53 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.357591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583437-6094-483d-83ec-c1cee4103e53 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.363749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 370/730] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:46.365212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583437-6094-483d-83ec-c1cee4103e53 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.389852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.390401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.390678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.390948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.400031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.401476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7a2e3616-44a5-4a89-89cf-cfcd44fee4e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:46.412974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:46.413986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:46.415019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:46.419004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 361/731] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:46.420293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.420576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.420848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.440051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:46.441329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:46.466747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.474369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.501235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583437-6094-483d-83ec-c1cee4103e53 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.519783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 362/732] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:46.519783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.556004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.557131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.557412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.557682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.568421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.569881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:46.584578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:46.585586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:46.586493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:46.586493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:46.586493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:46.590951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 371/733] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:46.592104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.651990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3a4d-872a-4d97-8445-a0dea4770391 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.686964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.687561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.687863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.688128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.688833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:46.689135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.690538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7267d-1b4f-4de5-844c-1647a2a158ea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.697964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c2741-cf42-4fde-bace-90ff8f693012 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.706963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 363/734] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:46.708130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463df95-51bc-4972-a97e-aebf76be5c04 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.708404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463df95-51bc-4972-a97e-aebf76be5c04 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.708671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463df95-51bc-4972-a97e-aebf76be5c04 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.709834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 364/735] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:46.710509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463df95-51bc-4972-a97e-aebf76be5c04 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.727504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.737940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.738330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.747930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.747930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.747930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.747930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.747930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f60f4-b24b-4279-8406-fe08dc998f55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.808681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463df95-51bc-4972-a97e-aebf76be5c04 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:46.840750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:46.841582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.841864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.842132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.842754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:46.843054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:46.844435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9564db-8593-44fe-9fa8-2e64b4ccce04 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:46.855786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 365/736] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:46.857222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aa5d-2136-4f63-94ee-279707c00f64 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.857498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aa5d-2136-4f63-94ee-279707c00f64 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.857769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97aa5d-2136-4f63-94ee-279707c00f64 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.858941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 366/737] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:46.859616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aa5d-2136-4f63-94ee-279707c00f64 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:46.868604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 372/738] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:43:46 2022] POST /identity/v3/services => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:46.870142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:46.870668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:46.871043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:46.897424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:46.898627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:46.960167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:46.984343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97aa5d-2136-4f63-94ee-279707c00f64 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.000022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 373/739] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:47.000022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.024934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.025798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.026070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.026337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.099084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.100600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.113707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5238d-59f6-47cb-acd3-a2679f91c729 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.127719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.149517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.150354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.150630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.150901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.158946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.160398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ef476cd2-f345-4a2a-8642-f1ba606262c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:47.173709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:47.180047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 374/740] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.181247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.182130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.194635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.195141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:47.196337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:47.215656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 367/741] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 10:43:47.216731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.273809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 375/742] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 10:43:47.275398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.335021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc037d8a-88ba-45e3-b73c-b9aab4ff9567 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.372257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.373212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.373681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.374147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.379097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f0560-1ba1-4510-874f-cf3827d12b1f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.380814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.385052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:47.389681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:47.401065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 368/743] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:47.402131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.424681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.425279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.425559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.425886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.426609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:47.426918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.428352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.440927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.450197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 376/744] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:47.450197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.511707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59d6a-f42f-44ae-89df-c8bef01b76df None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.548238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.548887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.549165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.549454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.550483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05776b-af72-435f-9a7f-2d413b77bec6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.561126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.562688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:47.578559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:47.579560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:47.580503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:47.580503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:47.580503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:47.580503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 369/745] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:47.580503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.593497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.594098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.594372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.594635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.595269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.596679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.683797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 377/746] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:43:47 2022] POST /identity/v3/services => generated 240 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:47.683797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.709348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.745782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.753833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa70880-0a87-4957-8252-f8ac60f025ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.797156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.797691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.798319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.798609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.799554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:47.799886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.808204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.808605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35391d-5938-4428-8568-0be051e31d7f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.813710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 370/747] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:47.818790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.853166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.853751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.854015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.854280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.865254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.866783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:47.882573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:47.883738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:47.885231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:47.885231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:47.885231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:47.885231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 378/748] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:47.886245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:47.946164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf125ec-9a9f-4d70-b0bf-396617499233 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:47.977410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:47.984346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:47.985030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:47.985366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:47.985706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:47.986407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:47.986793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:47.988267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.003702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 371/749] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.006059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.018388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba08367-8003-4db3-bc37-bca6d1c6a3ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.029039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.057715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.058260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.058536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.058802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.070745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.072855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:48.089887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:48.090873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:48.092082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:48.092082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:48.092082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:48.092082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 379/750] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:48.092760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.141982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ee5d7-2e71-4444-82f0-449dd4383709 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.186933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.187671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.188075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.188438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.189670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:48.190068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.192127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.201816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae74603-6704-4210-874c-1c5da41836bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.211850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 372/751] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.213861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.232140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.247853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.248634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.248916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.249187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.260476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.262098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:48.275376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:48.276325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:48.277242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:48.281284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 380/752] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:48.282105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.310622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.349988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e99d7-576c-46c0-a047-e1a4eaeb7154 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.394965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.395208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88832f-7a06-468d-9ff2-98fab1192ea0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.396080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.396800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.398147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.398989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:48.400101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.401673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.418593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 373/753] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.420482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.434833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.435825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.436174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.436577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.443982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.445740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:48.458714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:48.459692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:48.460675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:48.464201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 381/754] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:48.465212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.516333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.551438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5ac3c-bf3e-4f50-9871-844a2bb1a4fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.583219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e8c6c-a922-436b-9bca-4a0fecfb4414 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.592517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.598471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.599171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.599376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.599614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.600573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:48.601546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.602515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.620329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 374/755] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.622093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.627254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.628144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.628598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.629041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.639417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.641340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:48.656283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:48.657258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:48.658217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:48.661230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 382/756] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:48.662162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.734343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5ffaf-aa52-4148-8cbb-19a2539871a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.768175 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3dce2-443a-4122-bdc5-8920e923919f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.771808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.773220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.773892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.774267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.774662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.775458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:48.775855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.777343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.795314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 375/757] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.796376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.806520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.807395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.807893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.808362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.809259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=8bf57a090d4f4ce5b950ccb1c1c584a6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:48.809771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.811651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:48.817843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 383/758] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/role_assignments?user.id=8bf57a090d4f4ce5b950ccb1c1c584a6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:48.818914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:48.892531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:48.932490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9705fbc-99f1-4724-800e-84fc67888853 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.971852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495bfb8-e150-484b-be38-17ceecd2301d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:48.974961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:48.975793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:48.976082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:48.976359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:48.978494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=d622a3b3cc2a4e469033cb2d5c66d2ed, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:48.979980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70b382-d424-43d2-bf51-45e0a2aef10f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.012297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.013293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.013768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.014233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.015218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:49.015801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.017503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b310ede-fc4f-4ef7-b549-17fb952e9160 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.037714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 376/759] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:49.039391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369a8a-52b1-46e3-9437-129aeb85b04a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.039668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369a8a-52b1-46e3-9437-129aeb85b04a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.040001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2369a8a-52b1-46e3-9437-129aeb85b04a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.043622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 377/760] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:49.044492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2369a8a-52b1-46e3-9437-129aeb85b04a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.080099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 384/761] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:48 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:49.081559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.083837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.083837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.124940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:49.126135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:49.168754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.170889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2369a8a-52b1-46e3-9437-129aeb85b04a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.187237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.231077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 385/762] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:49.238560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.239791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.240309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.240808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.264956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.321113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.323495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.401741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bb741-e338-4927-8f69-217d27d3a83e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.427813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 378/763] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 10:43:49.435395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.449696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.450324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.450598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.450865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.451448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.452825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.456158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:49.467897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:49.471522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:49.472402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 386/764] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:49.478954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.529029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.576125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33543f1d-2581-44fa-8fb4-b7c93be8dc74 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.635674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.636333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.636608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.636938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.657326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.660052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:49.686859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:49.687870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:49.688766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:49.688766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:49.688766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:49.688766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 379/765] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:49.697411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.703362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772295c-c4f7-45c9-896b-7197e00df6b1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.739543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.740572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.741046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.741526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.742410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:49.742705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.744647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.751268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 387/766] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:49.755877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.804145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.867566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7b24b-1fb7-4b16-ba0e-a2f4a710d7ac None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.902832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:49.911025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.911636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.911945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.912218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.912855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:49.913155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.914521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a2ab8-9e70-43a5-a93b-672d05a6e1e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.927583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184793f-eb7f-46ff-b8cc-669b929cfcae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:49.932724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 380/767] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:49.934064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328ded-00e4-48b9-97e3-e502318b3119 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.934347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328ded-00e4-48b9-97e3-e502318b3119 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.934620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59328ded-00e4-48b9-97e3-e502318b3119 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.936338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 381/768] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:49.937106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59328ded-00e4-48b9-97e3-e502318b3119 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:49.975117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:49.976171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.976457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.976734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:49.977371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:49.977708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:49.979070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff26d6-85ca-4999-aaf0-d5ab5e5d34a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:49.990268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 388/769] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:49.991545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:49.991853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:49.992425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.010868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:50.012238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:50.072035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59328ded-00e4-48b9-97e3-e502318b3119 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.087172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 389/770] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:50.088335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.105936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:50.117763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.118531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.118812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.119136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.126134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.127681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:50.140294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:50.141229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:50.142133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 382/771] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:50.142623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.196778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:50.225495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f5de-ac89-4fb0-bc5a-28766b9d9920 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.284456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10279db0-d09f-4ea7-9fc7-61a5ecbc9ca5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.295521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.296182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.296464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.296736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.298602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] RBAC: Adding query filter params (interface=public, service_id=0c052f62067c4103870b83dd1876c807, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:50.298918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.300309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.323963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 390/772] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/endpoints?service_id=0c052f62067c4103870b83dd1876c807&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:50.324843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.363166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.364210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.364210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.367184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.367184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.367184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:50.384892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:50.385908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:50.386765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 383/773] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:50.387566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.484150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:50.531352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa3bb7-0b65-4f23-9fdb-0586b4f0010f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.566492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c539b-a80b-40f3-bbb6-7896f5e993ed None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.609069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.609691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.609976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.620381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.620381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.620381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.631787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 384/774] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/services => generated 772 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:50.631787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.632043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.632043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.632043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.632043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.639792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:50.639792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.640079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.687650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 391/775] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:50.688606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.704296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:50.868384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b898393-fe0c-4828-9c81-1adba386abd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.876178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2899d-8ecd-4ebf-8dc2-5b3539631ce6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:50.881840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:50.910431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.911045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.911324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.911601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.912286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:50.912835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.915025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.917248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:50.917855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:50.918128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:50.918393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:50.919173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d0dea5e1e47b48b0b7623c6b7100279e) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:50.919482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:50.920661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 385/776] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:50.920924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b592b57-9a9f-4ff4-a414-d290068eaa3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:50.926429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:50.929235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 392/777] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/role_assignments?user.id=d0dea5e1e47b48b0b7623c6b7100279e&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:51.058404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920abddc-d87f-4f38-9578-0ce6873f165f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.109942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.111029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.111532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.112021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.112852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:51.113365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:51.114927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:51.124388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 386/778] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:51.124893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832d14-4954-49c4-b15a-1802055cdc8f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.125243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832d14-4954-49c4-b15a-1802055cdc8f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.125522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35832d14-4954-49c4-b15a-1802055cdc8f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.126954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 393/779] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:51.129480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.133385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.133744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.134027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.161637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:51.162830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:51.209053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:51.303705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:51.314113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 394/780] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:43:51.334539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594eb42-cdbc-4681-a411-a6789dc4095b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.382153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.382733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.383012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.383276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.384063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] RBAC: Adding query filter params (interface=public, service_id=162e0d4e528d4bc492bf04003e4ded72, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:51.384368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:51.385731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:51.398262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 387/781] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/endpoints?service_id=162e0d4e528d4bc492bf04003e4ded72&interface=public®ion_id=RegionOne => generated 194 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:51.399818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.401166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.517087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a00fc7-249f-4265-8b1d-743ad2ebfdfc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.518297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed72380-6827-4b82-95a9-75a1142dc732 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.519951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:51.553028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.553648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.553971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.554248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.554866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.554995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:51.555858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.556275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.556410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f9a4-5478-4314-9fa9-3551c3415b1c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:51.556802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.567955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 395/782] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:51.569418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc194e-ed51-458a-8564-92a17f4ee1a5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.569702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc194e-ed51-458a-8564-92a17f4ee1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.569979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc194e-ed51-458a-8564-92a17f4ee1a5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.570089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:51.572184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 396/783] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:51.576609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:51.584852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:51.585927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:51.586806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.586806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.586806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.587111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 388/784] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:51.587111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.628906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:51.630194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:51.706081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2a3c-10f7-4d71-8bf9-408fdaf83e68 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.748900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:51.767312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.767965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.768248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.768524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.769200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:51.769505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:51.770897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:51.775436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 397/785] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:43:51.783793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 389/786] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:51.789511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.817630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:51.848054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:51.927829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dcdd0-3019-4970-8f22-69e31ed8f56c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.934819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf2cb-6377-47a1-b868-4e92ad17612a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:51.972077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.972811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.973176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.973458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.973624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:51.974193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:51.974467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:51.974736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:51.984310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:51.986151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:52.002660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:52.004864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:52.006921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:52.006921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:52.006921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:52.006921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 398/787] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:52.007436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:52.009948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:52.010918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:52.011802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 390/788] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:52.013604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.020799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.065985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:52.130435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8fb8e-9556-43a3-8f17-19a28dd12628 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.170785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.171673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.172271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.172692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.173641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:52.174091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.175603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.191101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 399/789] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:52.197439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.222852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:52.241539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0ea2c-bfaf-43b8-8635-446f264ab1f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.285962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.286562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.286833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.287092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.287740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:52.288057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.289398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.316128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 391/790] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:52.329073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.418393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8578266-3c47-4254-93a2-68fc7e197e59 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.425668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:52.460867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cd975-57b8-41d2-8329-67f17a42b07a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.499400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.499982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.500370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.500729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.512482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.512817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.513122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.513397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.513667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.514360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.521263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:52.530510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:52.532548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:52.533445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:52.533445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:52.533445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:52.533445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 392/791] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:52.537315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.537518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.538775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:52.564879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:52.565899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:52.566853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:52.568750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 400/792] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:52.573751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.652373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4298ea-896c-47bf-be23-694520b3ec4a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.688577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.689344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.689624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.689898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.690538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:52.690838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.692234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.710585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 393/793] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:52.712416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.780273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:52.822568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445de938-21c5-4e15-acf1-5198116a34ad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.860017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:52.860685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:52.860962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:52.861286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:52.869929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:52.871387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:52.883794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:52.884760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:52.885702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 394/794] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:52.890758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:52.957669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3117f0-6466-43f6-b1eb-d87a90a4d0df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:52.998240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d3d01-11eb-4f6e-ae75-857116a8b202 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.021877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.022492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.022743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.022850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.023023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.023849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:53.024224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.029896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.052652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.053208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.053520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.053798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.058036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:53.058352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.059792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a26efb-f333-4017-8c76-c7b4339de47e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.067412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 401/795] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:53.073755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.088892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 395/796] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:43:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:53.090151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe45dea-b072-4b33-a285-a4b800a8d90e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.090438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe45dea-b072-4b33-a285-a4b800a8d90e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.090713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe45dea-b072-4b33-a285-a4b800a8d90e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.097081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 396/797] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:53.097841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe45dea-b072-4b33-a285-a4b800a8d90e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.107497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.206385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce01f9-d5e9-41ad-973c-87195aff2dc5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.222212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe45dea-b072-4b33-a285-a4b800a8d90e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.240474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.241091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.241365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.241631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.251424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.252873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.264181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.264778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.265055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.265323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6ef02eca-94f4-46ca-b5c9-c7ca15d93f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:53.267308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:53.268292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:53.269195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:53.269195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:53.269195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:53.269195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 402/798] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:53.269834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867de79-6d98-4919-bfce-c57893d3807b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.269937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867de79-6d98-4919-bfce-c57893d3807b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.270128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867de79-6d98-4919-bfce-c57893d3807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.288466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9867de79-6d98-4919-bfce-c57893d3807b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:53.290140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9867de79-6d98-4919-bfce-c57893d3807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:53.313611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.336822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.338264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b056538-aabf-48f3-90d6-ee67d853417e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.360168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 403/799] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:53.361414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d600b51-0cae-486f-b3ce-08414bd69ff0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.361696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d600b51-0cae-486f-b3ce-08414bd69ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.361969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d600b51-0cae-486f-b3ce-08414bd69ff0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.363337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 404/800] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:53.364543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d600b51-0cae-486f-b3ce-08414bd69ff0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.373535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.460766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 397/801] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:43:53.461994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.462572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.462572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.473994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d600b51-0cae-486f-b3ce-08414bd69ff0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.480611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:53.481794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:53.512675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.513299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.513560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.513828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.514457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:53.514759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.516147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.528074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 405/802] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:53.528680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.559414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 398/803] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:53.560622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.574071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.609449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.647684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c6aea-5e3f-4071-932d-05e9ca4ff9ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.685049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544db14-b5aa-4e44-b7ba-484b685e7bad None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.685483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.686117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.686442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.686714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.687343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.688753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:53.698436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:53.699473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 10:43:53.700401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:53.704517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 406/804] 10.222.0.48 () {56 vars in 1129 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:53.705188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.729094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.729722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.730006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.730271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.744254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.745710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:53.765373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:53.766338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:53.767258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:53.767258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:53.767258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:53.771660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 399/805] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:53.772511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.811090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b891-115a-4831-93bc-b7ae9cef1852 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.834655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.839124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:53.847074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.847723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.860903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.860903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.860903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:53.860903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:53.868230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:53.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 407/806] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:53.870056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.884624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a34a8-aa76-4c4a-8994-5c290936354c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:53.920589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:53.921216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:53.921496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:53.921763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:53.933426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:53.938643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:53.939569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:53.940463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 400/807] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:53.941199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:53.982303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bd2d4-7d91-4a69-83a4-3e063a572deb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.017645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.018269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.018525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.018793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.019414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.019712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.021094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.025679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 408/808] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.026443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.053125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0122-ddcc-4133-b500-3db2aa1a72f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.070112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.081255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.087276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.087847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.088138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.088404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.089037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.089335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.090682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.102065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 401/809] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.103091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.128087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe2490-eff6-4559-bf2a-b9054888acf0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.162216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.162805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.163081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.163356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.164049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.164353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.166329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.178062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 409/810] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.178828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.203664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c7fc6-0286-48c2-916f-4efb1372fdc4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.236263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.236859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.237133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.237405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.238073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.238374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.239723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0d421-2d2c-44c3-8d1b-6a862e6aea48 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.248179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 402/811] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.249314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b0df3-b1e0-4e5a-8321-3aa711b42cc1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.249594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b0df3-b1e0-4e5a-8321-3aa711b42cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.249869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b0df3-b1e0-4e5a-8321-3aa711b42cc1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.254923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 403/812] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:54.254923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b0df3-b1e0-4e5a-8321-3aa711b42cc1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.276899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9fa4d-c300-486c-a5a8-6db0ff98ff1e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.289716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.294342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.309994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.310582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.310854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.311122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.311737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.312059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.313414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.317946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 410/813] 10.222.0.48 () {56 vars in 1124 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.318684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.353679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b0df3-b1e0-4e5a-8321-3aa711b42cc1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.387063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.387685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.387988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.388259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.394655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.396105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:54.407372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:54.408396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 404/814] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:54.409280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.420325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5fb4-0470-4f2b-b92d-18f58c6c8e88 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.453157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.453794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.454066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.454332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/8bf57a090d4f4ce5b950ccb1c1c584a6/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.468268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=8bf57a090d4f4ce5b950ccb1c1c584a6, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.469756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193f725-9ce8-4be1-8c03-148ecfb1a385 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.504174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.507007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53a75-fd28-45b0-8fcb-edf2704e6835 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.539027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.539686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.540037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.540309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.546349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 411/815] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:54 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/8bf57a090d4f4ce5b950ccb1c1c584a6/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:54.553835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.553835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.553835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.558444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.559975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.568016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:54.568926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:54.577286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:54.584515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:54.585595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:54.585595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:54.585595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:54.585595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 405/816] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:54.585595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.607388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.675836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 412/817] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:54.675836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.705338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78061866-cccb-4467-b672-efe34e5be57d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.743922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.744557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.744831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.745097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.745746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.747115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.786832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef84bb-67c9-4d11-8f5a-b38e1a836b6a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.821926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:54.822519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:54.822796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:54.823064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:54.823741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:54.824068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:54.825474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:54.837321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 413/818] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:54.838104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.858614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 406/819] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:43:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:54.859471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:54.860129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.875542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:54.965365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26f263-a6f9-49c6-8f15-29614570d1cf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:54.996114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dfe83-7ede-4c0e-abce-88b40a50e371 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.004141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.004828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.005159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.005428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.006290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.006470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.007911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.021710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 414/820] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:55.022629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.045107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.046415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.046699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.047137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.047721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.049166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:55.060045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:55.061246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:55.063088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 407/821] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:54 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:55.068358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.121710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.143573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.153440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dd7a4-823b-471e-98bd-523546b15ba9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.176770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c160-ad7f-4cfe-82a7-8382acb9f2cc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.202806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.203932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.204360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.205040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.206337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.207040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.209438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.217754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.223933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.223933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.223933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.224264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 415/822] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:55.224264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.240765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.242450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:55.272947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:55.274066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:55.275046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 408/823] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:55.280222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.378305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c998-554d-4bb9-9a12-944330ed5743 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.414205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.415200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.415478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.415744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.416399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] RBAC: Adding query filter params (name=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.416698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.418043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.423101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 416/824] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:55.428732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.432733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.434826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad9951-6ab7-4a66-909d-3397b39cb763 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.483471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.496395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.497265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.497551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.497983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.499028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.499601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.501108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.525083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 409/825] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:55.532797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.546555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0540e-c958-404e-a335-163d36a48703 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.602450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.603436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.603876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.604289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.605007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.605474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.607423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.614351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 417/826] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:55.619970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.661041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f691367-7c9d-41b4-b6c6-260b2df410b1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.703413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.703900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.704923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.705351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.705766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.728293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.730079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:55.754323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:55.755573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:55.756619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:55.756619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:55.756619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:55.761657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 410/827] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:55.768201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:55.775815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafe025-62ac-4e9c-b45a-9f3ee607add4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.839357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.840236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.840653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.841341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.842133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.843613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.854816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:55.916342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1a907-067c-440e-ae86-f39de3ab8360 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:55.925740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 418/828] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:43:55 2022] POST /identity/v3/endpoints => generated 337 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:43:55.976019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:55.976666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:55.977599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:55.977870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:55.978502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:55.978796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:55.980216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:55.997727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 411/829] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:56.010291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:56.146649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115003e-bef4-481e-9f37-4acb9a80eaa4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:56.175841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:56.190163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:56.191181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:56.191669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:56.192214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:56.201444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:56.203184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:56.217278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:56.218220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:56.219205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:56.221661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 419/830] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:56.227506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:56.336720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:56.344224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe37f2d-1156-4444-8526-c4137a1a7b9e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:56.383015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:56.383731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:56.384033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:56.384303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:56.385027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:56.385279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:56.386702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6fb23-511d-4502-a39f-71899cfb59af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:56.401391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 412/831] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:56.407222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:56.526843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de0d-fbfc-421c-8be9-491635b5f727 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:56.560639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:56.561273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:56.561547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:56.561816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d0dea5e1e47b48b0b7623c6b7100279e/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:56.563890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=d0dea5e1e47b48b0b7623c6b7100279e, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:56.565364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea42ec-3635-40df-8310-066725aff4ae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:56.624202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 420/832] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:43:56 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d0dea5e1e47b48b0b7623c6b7100279e/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:43:56.760953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5b13-32bb-4026-a26c-859f272ed0a3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:56.761988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5b13-32bb-4026-a26c-859f272ed0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:56.762492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5b13-32bb-4026-a26c-859f272ed0a3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:56.764367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 413/833] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:56.770240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:56.770525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:56.770804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:56.790703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:56.791821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:56.838435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:56.865933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 421/834] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:56.890523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5b13-32bb-4026-a26c-859f272ed0a3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:56.993738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe5b13-32bb-4026-a26c-859f272ed0a3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.027715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.028701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.029181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.029700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.041260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:57.046245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:57.047172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:43:57.048070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:57.050587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 414/835] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:43:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:57.057004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.161873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3cce-9425-4e5b-abae-0d12ce94976c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.197304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.197896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.198180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.198445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.199112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:57.199412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.200794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b278-2f87-4e2a-a8ba-4edbb2d3e82e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.209441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 422/836] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:57.215436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.232455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.252552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35c31-d5a6-4cb1-95b3-c107fc051572 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.252836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35c31-d5a6-4cb1-95b3-c107fc051572 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.253126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe35c31-d5a6-4cb1-95b3-c107fc051572 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.254780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 423/837] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:43:57.259939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.260387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.260480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.280818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:57.282324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:57.323520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.326736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de349cd-f27d-4a8e-a855-4b7f888bb370 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.367018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.368771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.369254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.369725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.379095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 424/838] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:43:57.382561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.384817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:57.403459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:57.404500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:57.405390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:57.405390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:57.405390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:57.405390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 415/839] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:57.405626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.412730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.486194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.526982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a10078-368c-4cde-9c4c-2431d27cae39 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.528460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e2a7-ca0c-4b6e-9c05-5d19378910d4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.568329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.569314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.569793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.570265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.571711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:57.572282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.572763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.573399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.573727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.573948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.574530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.575525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.576007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:57.587418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:57.588482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:43:57.589346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:57.596895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 425/841] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:57.596895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.599157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 416/841] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:57.599157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.638325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.710801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ccbd6-7a5a-4146-ae51-4ad43b6e1490 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.729284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b332-3eb2-4a54-9f64-621c821dce1c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.751653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.752875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.753168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.753718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.754377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:57.754717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.757182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7131d35-d2c6-478e-9000-ee986a7c9879 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.758528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.765621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 426/842] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:57.766942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f499c-9186-4d78-9e42-48664ee837b1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.769136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f499c-9186-4d78-9e42-48664ee837b1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.769136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f499c-9186-4d78-9e42-48664ee837b1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.769136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 427/843] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:57.772949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.773429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f499c-9186-4d78-9e42-48664ee837b1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.774269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.774694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.775108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.782698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.784412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e9edd9db-6ebd-47c8-bab4-2a84c473819a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:57.796338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:57.797424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:43:57.798580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:57.800677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 417/844] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:57.801976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.802259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.802537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.822514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:57.823880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:57.873695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:57.881080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f499c-9186-4d78-9e42-48664ee837b1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:57.898119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 418/845] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:43:57.899482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:57.932069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:57.933231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:57.933714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:57.934182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:57.935183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:57.936183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:57.937543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:57.951794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 428/846] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:57.951794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.023702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c770e-224b-4fe5-8bbc-ae7c99b18f5d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.053385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6233c-e58c-4d61-9e4a-4e447fc374d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.053920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.072702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.073637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.074432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.074723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.075541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:58.075983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.078213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.095138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.095927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.096271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.096631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.097285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.098725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.107080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 419/847] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:58.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:58.112867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:43:58.113996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:58.114895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 429/848] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Aug 11 10:43:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:58.116434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.120744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.150000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.235238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e874b-affd-478e-a4ae-d180144afffa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.253540 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950b06f-cdfa-45e0-b168-cd91af2ae483 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.279010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.280255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.280774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.281571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.290290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.290980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.291381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.291722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.292507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:58.292875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.294794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.305266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 430/849] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:58.306323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.343444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.370035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.371518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77848865-bfb6-4af3-9c57-97293b0c205f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.403072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.420032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9903e7-4c17-41aa-b69d-87728ebebdc5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.458145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.459212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.459687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.460185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.461031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:58.461645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.463288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.476271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 431/851] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:58.480098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 420/851] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:43:58.482010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.595871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb709abf-a0cf-4a8a-8a1f-84090e689167 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.639945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.640947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.641453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.641933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.653546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.655333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:58.671246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:58.672421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:58.673492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:58.673492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:58.673492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:58.673835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de654-8a69-4ea3-9dbf-1d953fd8b5c2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.673835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de654-8a69-4ea3-9dbf-1d953fd8b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.673835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9de654-8a69-4ea3-9dbf-1d953fd8b5c2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.674040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 432/852] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:58.677053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 421/853] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:43:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:43:58.678262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.681451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.683203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.683483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.683789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.704185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:43:58.705375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:43:58.779353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 422/854] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:43:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:43:58.802214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.806762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.812208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5282c9-ec13-4656-8b6b-0eb30a4ae65f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.857762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.858284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.858564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.858833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.859472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:58.859791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:58.861976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:58.878730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 433/855] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:58.884231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.913819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b443623-2b20-4264-b1e4-f62ed59b1103 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:58.945684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:58.953198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:58.957791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:58.957791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:58.957791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:58.967584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:43:58.972718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:58.973644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:58.974786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 423/856] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:58.982685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:58.986985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b334b-e995-45e2-94f6-1ab8e42db97d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.033363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.034045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.034324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.034603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.042333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.043798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:43:59.056037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:59.056959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:43:59.057862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:59.061846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 434/857] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:59.062463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:59.062986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.112118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d3f08-4828-412a-a41d-0db0fe7bd231 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.152155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.153217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.153683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.154125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.155031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:59.155640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.157307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.172914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3a8dd-ece0-4560-8aee-beab3a6d11c8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.173303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 424/858] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:59.174800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.195340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:59.206751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.207349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.207688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.207988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.208675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:59.208999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.210396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.223448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 435/859] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:59.229573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.292896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2abe1-aa3a-40ce-8eab-2550704e292a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.319946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:59.336759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.338266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.338694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.339108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.356153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c609-2142-4a38-9ebc-1799d412e889 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.361369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.364499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:43:59.388914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:59.390096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:43:59.391181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:43:59.391181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:43:59.391181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:43:59.393372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 425/860] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:43:59.399727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.402938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.403716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.405054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.405489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.406360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=8bf57a090d4f4ce5b950ccb1c1c584a6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:59.406822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.408423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.424213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 436/861] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/role_assignments?user.id=8bf57a090d4f4ce5b950ccb1c1c584a6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:59.522215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8c7d8-0961-4bf2-b7b8-6aa525c631ad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.560561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.561173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.561445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.561755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.562406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:59.562703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.564146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.582362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 426/862] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:43:59.584868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.635248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:59.698669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c7a5b-9c5e-49b7-a2ee-7e9e3f802d97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.737143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.738022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.738477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.738891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.747509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.749196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:43:59.761411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:43:59.762387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:43:59.763592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 437/863] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:43:59.770288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:43:59.814966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:43:59.876315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c0596-54ed-4c19-80c2-1c3faf66a5c5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:43:59.910690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:43:59.911212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:43:59.911489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:43:59.911774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:43:59.912499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:43:59.912802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:43:59.914746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0809a65-545d-44ab-a183-7833828fe89b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:43:59.935778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 427/864] 10.222.0.48 () {56 vars in 1135 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:43:59.941493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.042453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082251-50b4-4f8a-bd8e-bb5a872d4a9b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.042805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082251-50b4-4f8a-bd8e-bb5a872d4a9b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.043002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82082251-50b4-4f8a-bd8e-bb5a872d4a9b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.044706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 428/865] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:00.045773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df4a96-78b3-46d5-8862-090966658a8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.049704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03103-058f-448c-88e8-65302e75c693 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.049982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03103-058f-448c-88e8-65302e75c693 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.050250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b03103-058f-448c-88e8-65302e75c693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.056765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.069022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01b03103-058f-448c-88e8-65302e75c693 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:00.071343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01b03103-058f-448c-88e8-65302e75c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:00.088458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.089185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.089839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.090458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.154650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 429/866] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:00.177150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b03103-058f-448c-88e8-65302e75c693 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.177926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.180435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:00.210504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.305420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b03103-058f-448c-88e8-65302e75c693 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.309157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 438/867] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:43:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:00.314563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.323105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.352022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.352895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.353322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.353761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.354573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.356532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:00.429131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba2f1b-e03f-4a7e-9e24-2f0926322310 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.433833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 430/868] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:00 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:00.477142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.478087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.478513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.478935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.491411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.493277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cb918dd1-4588-41bb-9df0-868561064003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:00.510443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:00.511665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:00.512848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:00.512848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:00.512848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:00.514685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 439/869] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:00.520726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.623174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.639560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894463-75b8-404c-8d0d-6e7806ffd899 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.640518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894463-75b8-404c-8d0d-6e7806ffd899 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.640791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36894463-75b8-404c-8d0d-6e7806ffd899 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.641151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80540ca-0d5e-497f-891a-98b932e4e455 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.642262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 440/870] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:00.647705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.648140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.648393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.668313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:00.674288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:00.683010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.684254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.684740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.685538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.686869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:00.687659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.690976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:00.707810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 431/871] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:00.719634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.726364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.745240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 441/872] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:00.769135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.848811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2577-1a2d-465e-9216-c89ddeb6406f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.885756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.886859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.887335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.887892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.890529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015495-3075-4fb9-9e69-f62ddc26c526 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:00.895398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.897108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:00.909459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:00.910383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:00.911314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:00.918524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 432/873] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:00.919621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:00.928481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:00.929023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:00.929152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:00.929350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:00.930142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:00.930831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:00.932650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:01.016480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 442/874] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:00 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:01.028143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075bc3-5f4b-4438-b11a-9179c95e0c8a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:01.072898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:01.073512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:01.073853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:01.074127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:01.074821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:01.075124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:01.076623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaec9c-419b-4e0d-b443-52fadcbe6399 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:01.091331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 433/875] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:01.097657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:01.154934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:01.214635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923a789-9280-4a46-b730-1bd0c2edafb2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:01.250743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:01.251344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:01.251623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:01.251919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:01.252634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d0dea5e1e47b48b0b7623c6b7100279e) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:01.252938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:01.254351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaefc74-2a2a-41f0-ab9f-4018d7b79bc3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:01.261420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 443/876] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:01 2022] GET /identity/v3/role_assignments?user.id=d0dea5e1e47b48b0b7623c6b7100279e&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:01.709538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f029-a2cc-450f-817d-dfbb11a34fc2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:01.709816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f029-a2cc-450f-817d-dfbb11a34fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:01.710094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7f029-a2cc-450f-817d-dfbb11a34fc2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:01.712324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 434/877] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:01.717679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:01.718123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:01.718263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:01.737404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:01.738603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:01.815005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 444/878] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:01.837878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7f029-a2cc-450f-817d-dfbb11a34fc2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:01.900673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:01.939496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7f029-a2cc-450f-817d-dfbb11a34fc2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:01.979007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:01.979634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:01.979939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:01.980209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:01.990793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:01.996078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:01.997016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:01.998391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 435/879] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:02.003406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.045464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:02.107885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b36be0-e118-4daf-931d-956262d0b8b6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.142579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.143168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.143448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.143720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.144404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:02.144710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.146126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.154665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 445/880] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:02.160257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.269354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e540-0dd2-4539-8698-2567c53fddfc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.293562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:02.306149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.307018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.307455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.310866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.318277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.320044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dbb7fd0b-d010-4e1d-9287-255b64009ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:02.334508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:02.335452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:02.336908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:02.336908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:02.336908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:02.338935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 436/881] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:02.344541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.444591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:02.447920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae8507-9ec7-4294-b883-42716d595ece None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.483842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.484447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.484734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.484996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.485634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:02.485935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.487280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.497545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f235d1-d4bc-4dd5-b28d-8c39c014f5b5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.497835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f235d1-d4bc-4dd5-b28d-8c39c014f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.498107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f235d1-d4bc-4dd5-b28d-8c39c014f5b5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.499657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 437/882] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:02.500535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 446/883] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:02.505036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.505329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.505604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.505857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.525030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:02.526246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:02.618226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 438/884] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:02.635294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135d7c-8657-456a-9116-bedd95bb2ecf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.639650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.680789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.681692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.682566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.682996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.693619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.695417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:02.713632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:02.714805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:02.716288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:02.718941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 447/885] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:02.727522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.748714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:02.755512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d185fe-db1f-4f07-ba23-b85737bec508 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.768089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:02.801721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.802360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.802636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.802922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.804207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.806294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:02.818553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:02.819571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:02.821136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 439/886] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:02.826629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.839180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd071-fb97-4e8b-8319-77e565263148 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.876672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.877275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.877553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.877885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.878751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:02.879055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.880595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:02.901223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 448/887] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:02.905925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:02.940622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560730e-da88-4f2d-b17f-e0c8e5471717 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:02.991042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:02.991570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:02.991928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:02.992202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:02.992829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:02.993130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:02.994497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.000086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 440/888] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:03.004802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.011627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.036999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f1468-fad6-47c9-af5e-9136c1cb4a7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.081989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.082596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.082875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.083146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.084169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.142539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbda22-0edc-4158-ad9e-6cf984f53790 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.165756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.167327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df80f0-9f5a-4585-bc46-2c8958796590 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.181236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.182163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.182689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.183162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.183985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:03.184466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.186191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.192213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 441/889] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:03.198803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.254703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.283078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 449/890] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:03.284682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd472e2c-e586-4a7a-930f-149a20775645 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.285120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd472e2c-e586-4a7a-930f-149a20775645 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.285552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd472e2c-e586-4a7a-930f-149a20775645 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.286850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 450/891] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:03.287539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd472e2c-e586-4a7a-930f-149a20775645 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.305328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.311125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2cf40-60ba-400a-a6c4-af2d70712681 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.346098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.346701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.346977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.347245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.348312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] RBAC: Adding query filter params (interface=public, service_id=abd9371b739145ba8d7f87b4688c3118, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:03.348312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.349659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da640eb-aa37-48e0-888d-0273ce370fb1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.355116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 442/892] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/endpoints?service_id=abd9371b739145ba8d7f87b4688c3118&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:03.356107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.356579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.356686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.374947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:03.376195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:03.400347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd472e2c-e586-4a7a-930f-149a20775645 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.436350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.436973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.437248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.437522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.447021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 443/893] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:03.447948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.448848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.450285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1a3bd941-e722-4fdb-acbf-16a1f288af04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:03.464666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:03.465918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 451/894] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1914-2549-447a-85e2-60ad0406002d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1914-2549-447a-85e2-60ad0406002d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1914-2549-447a-85e2-60ad0406002d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 452/895] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:03.471691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf1914-2549-447a-85e2-60ad0406002d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.500002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.537572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.546749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e2734-c0e1-483f-ba43-674e5cd3e740 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.569518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf1914-2549-447a-85e2-60ad0406002d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.580932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.581527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.586354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.586354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.586354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.586354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2811449-adf3-4b7d-ada3-c24e20f27979 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.603691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.604337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.604614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.604882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.605453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.606826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:03.615209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:03.621665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.622835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 453/896] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:03.623981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.640498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 444/897] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:44:03.640498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.640498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.640498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.656867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:03.658094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:03.732052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.738070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 445/898] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:03.738978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.752615 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26502e6a-ba5e-48d1-97fd-422c41740e9d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.794107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.794636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.794916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.795187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.795867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:03.796179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.797531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.810273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 454/899] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:03.811240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.836930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:03.853749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17491f9-b891-4ee5-a183-4ef9690ad2ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.890152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.890755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.891032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.891303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.891957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:03.892263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.894040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.899530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 446/900] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:03.900169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.920803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a2c4c-a7c5-40de-8229-6543490efb2b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:03.957622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:03.958250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:03.958578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:03.958848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:03.959451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:03.961425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:03.970423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:03.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:03.972550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 455/901] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:03.973390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:03.996618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.014000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a88f0-779d-44c2-a351-a7f14baf3760 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.049069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.049613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.049891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.050163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.057542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.058971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:04.070113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:04.071074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 447/902] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:04.072042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.083533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.087425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6873cc4-bac7-4997-b033-d0f8dd608fec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.125642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.126531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.126814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.127082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.127711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.128079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.129466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.134477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 456/903] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.135259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.189051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94a5fd-0ae6-4b3b-9cb0-6258d95c2df6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.216387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.226755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.227399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.227675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.227970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.228616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.228911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.230258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.239338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 448/904] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.239338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.244357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0c544-772c-4135-b732-7dcadf2c8ebc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.279017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.279711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.280098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.280374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.281051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.281351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.282761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.299360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 457/905] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.299360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.314315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.345215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eae8d5-6147-49c5-8e52-edcbcecac424 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.383063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.383675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.383975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.384251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.384948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] RBAC: Adding query filter params (interface=public, service_id=b4080a5f624d452c8ce21e967b0f3d05, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.385249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.386716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.396252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 449/906] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/endpoints?service_id=b4080a5f624d452c8ce21e967b0f3d05&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.396252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.406207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427a086-d6b9-46f8-bdaf-771be8df699d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.439594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.440304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.440641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.440975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.441660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.442019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.443433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.455798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 458/907] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.455798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:04.464016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.522589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731b63-6d76-405c-96cf-c2cee234e6f8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.575431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.576572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.577423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.578128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.579107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] RBAC: Adding query filter params (role.id=735e389132614971ad154ae4a9c80a7c, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=8bf57a090d4f4ce5b950ccb1c1c584a6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:04.579580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.579973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd2577-764b-4831-ab5f-2d75d8452948 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:04.581669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1a12b-5e13-42ce-a4b7-f467298ab5c0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.591935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:04.604112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 450/908] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/role_assignments?user.id=8bf57a090d4f4ce5b950ccb1c1c584a6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=735e389132614971ad154ae4a9c80a7c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:04.630851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:04.631508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:04.631902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:04.632103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:04.632801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:04.634187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45968a-d781-4f94-9256-fa9cac0afbf0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:04.641195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 459/909] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:04 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:44:05.669260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff6f8-ef88-4d70-a009-5460b3a0712d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:05.669819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff6f8-ef88-4d70-a009-5460b3a0712d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:05.669819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ff6f8-ef88-4d70-a009-5460b3a0712d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:05.671384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 451/910] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:05.676979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:05.677432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:05.677532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:05.696320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:05.697423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:05.764365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:05.769328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 460/911] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:05.792244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ff6f8-ef88-4d70-a009-5460b3a0712d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:05.900135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ff6f8-ef88-4d70-a009-5460b3a0712d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:05.934798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:05.935418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:05.935696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:05.935992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:05.936567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:05.937933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:05.947129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:05.948135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:05.949289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 452/912] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:05.954153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:06.060575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34444-47f7-4797-bc02-1942b3f06b8d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:06.097293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:06.097906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.098189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.098463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.099151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:06.099459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:06.100844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:06.105932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 461/913] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:06.111278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:06.178480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:06.217495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1ca0a-5e55-4dca-ba11-3613a671b92e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:06.252013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:06.252625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.252901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.253169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.253801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:06.254101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:06.255471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529ba9b-fd92-483c-9dfc-022f77945dd4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:06.261037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 453/914] 10.222.0.48 () {56 vars in 1114 bytes} [Thu Aug 11 10:44:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:06.266354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:06.343035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:06.386232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4cc0b-b561-43fa-b6f0-866133b0bb25 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:06.423871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:06.424475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.424754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.425028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.425696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:06.427055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c0c09-7564-4aa6-aba3-6a96b0fd4b11 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:06.502940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 462/915] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:06.582762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3889-d777-458e-b481-6ec0499f96bd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.583109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3889-d777-458e-b481-6ec0499f96bd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.583305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3889-d777-458e-b481-6ec0499f96bd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.591987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 454/916] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:06.596716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.597001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.597316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.628061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:06.629384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:06.702385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e9c57-1b60-4080-89f9-41c148d7c84a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.702907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e9c57-1b60-4080-89f9-41c148d7c84a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.702907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e9c57-1b60-4080-89f9-41c148d7c84a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.704576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 455/917] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:06.709812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a7a39-8975-42aa-9a0e-c97ca9b69155 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.710094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a7a39-8975-42aa-9a0e-c97ca9b69155 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.710923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a7a39-8975-42aa-9a0e-c97ca9b69155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.711691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 463/918] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:06.729662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ea3a7a39-8975-42aa-9a0e-c97ca9b69155 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:06.731069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ea3a7a39-8975-42aa-9a0e-c97ca9b69155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:06.736666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:06.787376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:06.817137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 456/919] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:06.818518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df830-821f-4695-b007-12dbdf11dcd4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.819168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df830-821f-4695-b007-12dbdf11dcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.819447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998df830-821f-4695-b007-12dbdf11dcd4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.822131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 457/920] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:06.830229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.830515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.831107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.846398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac76caf-94c5-43a8-81fe-ed22131637e1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:06.853476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:06.854684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:06.884121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:06.884867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:06.885151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:06.885433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:06.886079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:06.887440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:06.912254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:06.925230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 458/921] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:06.926038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:06.984135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 464/922] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:06 2022] POST /identity/v3/services => generated 241 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:06.985441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.058282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c3f95-ca1d-47b2-a229-9d1e12f57064 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.115847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.117651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.118158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.118992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.122035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10bb6-c077-4a0f-ad50-6f0b5d6a5631 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.131715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.134828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:07.146142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:07.147123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:07.148558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 459/923] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:07.156987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.164947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.165586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.165869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.166143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.166727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.168115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:07.177590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:07.178610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:07.180076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 465/924] 10.222.0.48 () {56 vars in 1132 bytes} [Thu Aug 11 10:44:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:07.185109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.214723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.271377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37629777-2459-4717-adc4-a928e2b49b06 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.304086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3b39-51ef-4ea1-b3ea-cd5811e7d9a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.306424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.307099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.307477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.310774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.311539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:07.312392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.313868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.324881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 460/925] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:07.332494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.351534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.352255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.352558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.352856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.353583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:07.354009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.355484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.361070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 466/926] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:07.366702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.415964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.464191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81bab1-5565-4cd6-a38c-aba2faa9216e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.474196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.501198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affbefb-053e-4b4c-b267-cd5c0011856f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.506777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.507994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.508272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.508543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.523147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.525266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:07.543173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:07.544451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:07.545626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:07.545626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:07.545626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:07.545626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 461/927] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:07.549730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.554757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.555627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.556161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.556645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.557582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:07.558088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.560288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.573513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 467/928] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:07.575169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.676232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1f101-94af-45c6-9f29-ca7e3d8698cd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.697978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.702587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdde3b6-7245-4bb7-a4dd-2d32f9c55bd6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.719294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.720157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.720504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.720842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.721566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:07.721926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.723375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.743039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 462/929] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:07.744173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:07.750844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.752100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.752527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.752954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.753768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.755960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.769611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:07.854686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2ed2b-afd2-4713-8845-8a1437820e7b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:07.859979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 468/930] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:07.905767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:07.906768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:07.907265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:07.907680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:07.916282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:07.918057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:07.932477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:07.933729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:07.934945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:07.937436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 463/931] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:07.945807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.058188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d667e-108b-4961-b9ed-794fe982b075 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.093541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.094142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.094438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.094680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.095394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:08.095695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.097199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.116024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 469/932] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:44:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:08.121580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.205372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:08.232498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666409bc-009c-4735-85d2-a22fab01cd9a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.268107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.268871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.269209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.269540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.280518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.282018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:08.299585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:08.300830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:08.302057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:08.302057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:08.302057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:08.303751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 464/933] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:08.309736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.325783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:08.417594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e82354-eaa6-46aa-adda-ef5f9c60cdfe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.452748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.453487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.453860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.454230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.454956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:08.455353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.456819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.470791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 470/934] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:08.476592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.580756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796385e-a856-4b82-ac42-b64a1a8e945a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.612148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:08.617704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.618333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.618675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.618935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.626110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.627563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e0da8445-13af-4451-bb68-d8336c4ce45c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:08.639451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:08.640443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:08.641687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 465/935] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:08.646378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.666782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81bd67-d3b4-49b1-8561-e2e5ad8ce7e4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.667093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81bd67-d3b4-49b1-8561-e2e5ad8ce7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.667335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81bd67-d3b4-49b1-8561-e2e5ad8ce7e4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.669053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 466/936] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:08.674760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.675321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.675860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.695331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:08.697004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:08.732161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:08.767988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e222e29-ab98-4846-b330-f10a677c474b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.799803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 467/937] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:08.817695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.818305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.818586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.818860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.820107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:08.820416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.821947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.824478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.835186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 471/938] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:08.844673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:08.930998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a9c1c-c147-42cc-ae2c-f423dc04ec80 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:08.949552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:08.968089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:08.968880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:08.969227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:08.969587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:08.970287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:08.971927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:08.974707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING keystone.server.flask.common [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 11 10:44:08.976249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19355a35-2e30-404c-9fd4-de753d7b585f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.013210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:09.018624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.019282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.019553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.019872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/8bf57a090d4f4ce5b950ccb1c1c584a6/roles/735e389132614971ad154ae4a9c80a7c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.021385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 10:44:09.021385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 10:44:09.021385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [parameters: {'id': '58959000d3654d84be962d3baf03eeba', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 10:44:09.021385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 10:44:09.023945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=8bf57a090d4f4ce5b950ccb1c1c584a6, role_id=735e389132614971ad154ae4a9c80a7c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.025505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da0aa1-a463-4493-a8a8-a997b6be6f36 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58959000d3654d84be962d3baf03eeba.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58959000d3654d84be962d3baf03eeba. Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:44:09.032762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 11 10:44:09.033649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Aug 11 10:44:09.034667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 10:44:09.035607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 10:44:09.036952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:44:09.043816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': '58959000d3654d84be962d3baf03eeba', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.046121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:09.049226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 58959000d3654d84be962d3baf03eeba. Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 468/939] 10.222.0.48 () {60 vars in 1140 bytes} [Thu Aug 11 10:44:08 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:09.051364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.119214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 472/940] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:44:08 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/8bf57a090d4f4ce5b950ccb1c1c584a6/roles/735e389132614971ad154ae4a9c80a7c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:44:09.135562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aad4a-0e82-4ae8-8b98-e23422a9dc67 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.135881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aad4a-0e82-4ae8-8b98-e23422a9dc67 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.136159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7aad4a-0e82-4ae8-8b98-e23422a9dc67 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.137733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 473/941] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:09.143521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.143825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.144105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.169565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:09.171047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:09.171199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5478fc-9d43-4bcf-b142-b4c497dde347 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.208828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.209472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.209752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.210550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.218912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.220895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:09.234371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:09.235590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:09.236717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:09.239842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 469/942] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:09.241848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.275632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 474/943] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:09.285397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:09.295864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.381100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d016d8e-d8de-479a-8740-a15ac70e3570 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.417507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c78e-146e-421d-8a5f-67f6dca99adf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.427150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.428458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.429472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.430267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.431450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:09.431940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.434288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.445054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:09.457156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.457786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.461384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.461384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.461384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.461384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.461938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 470/944] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:09.471950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:09.473322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:09.474345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:09.477497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 475/945] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:09.484009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.594591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:09.605417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861c4d0-94a8-4fba-81da-e5f578318868 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.646078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.646695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.646975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.647248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.647971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:09.648277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.649663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.655786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 471/946] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:09.660870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.771112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7326b-072d-487f-8494-d8660c95a79e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.815177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.816156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.816647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.817161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.817969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:09.818467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.820050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.825482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 476/947] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:09.831796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:09.935367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f1511-a000-47dd-98e3-1a043ed677cb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:09.971146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:09.971782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:09.972374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:09.972680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:09.973385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] RBAC: Adding query filter params (interface=public, service_id=b8f0895d2e6a4a25a5ab47e63ce1253a, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:09.973699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:09.975064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d005b0-8009-4842-b69b-6ce6a560d0ba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:09.980693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 472/948] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/endpoints?service_id=b8f0895d2e6a4a25a5ab47e63ce1253a&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:09.986140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:10.035187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb015-f16b-4bbe-b02a-a75845cbce6f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.035409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb015-f16b-4bbe-b02a-a75845cbce6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.035684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdb015-f16b-4bbe-b02a-a75845cbce6f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.037202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 473/949] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:10.042349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.042631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.042907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.046323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:10.068918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:10.070779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:10.089656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:10.104793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06669a-e46f-42f6-8839-d3144e9d08f7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:10.140902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:10.141506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.141791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.142064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.142668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:10.144053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:10.150919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 474/950] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:10.159872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 477/951] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:09 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:44:10.173104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:10.284640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046ea09-bab3-4c35-942e-44933bf21bba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:10.324357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:10.325284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.325619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.325894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.326478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:10.327874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:10.337211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:10.338476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:10.339430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 475/952] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:10.344655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:10.449317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233110d-22d6-4b23-9195-b9af0c95f0d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:10.463207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:10.483786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:10.484396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.484676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.484948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.485573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:10.485876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:10.487225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b974219-0a82-4c50-b400-7538a84b3066 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:10.492243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 478/953] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:10.497896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:10.515069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:10.607623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739671-9409-4897-9bea-d7bdd927b88f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:10.639190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:10.639828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:10.640113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:10.640386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:10.641013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:10.641314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:10.642682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4a4c6-efcf-4d33-b207-1e5395c0bcc5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:10.648278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 476/954] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:11.200665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098896-9fb7-48be-8732-666307ce8f2f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.201228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098896-9fb7-48be-8732-666307ce8f2f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.201228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2098896-9fb7-48be-8732-666307ce8f2f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.202813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 479/955] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:11.208037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.208448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.208615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.227681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:11.228879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:11.303876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 477/956] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:11.320530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2098896-9fb7-48be-8732-666307ce8f2f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:11.430651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2098896-9fb7-48be-8732-666307ce8f2f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:11.465838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:11.466427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.466718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.466989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.484318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-fdc56158-da8c-4e72-bb45-3fc08668ab64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:11.489107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:11.490102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:11.490971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:11.492943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 480/957] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:11.497356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:11.507830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:11.606195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321ac1a-ee97-458d-a923-744fb6fa269d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:11.640767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:11.641377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.641657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.641930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.642592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:11.642891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:11.644351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:11.646090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec561d-bb31-4b86-be64-e0f1a145c2e6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.646370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec561d-bb31-4b86-be64-e0f1a145c2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.646641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec561d-bb31-4b86-be64-e0f1a145c2e6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.649051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 481/958] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:11.654102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.654613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.655111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.659794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 478/959] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:11.663241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:11.680203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:11.681785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:11.722972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:11.742345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:11.778965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 482/960] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:11.779702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da691a6-c156-41dc-9e2e-8166710052be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:11.800313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:11.831600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:11.832270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.832551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.832827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.847417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:11.849336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:11.869218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:11.870264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:11.871181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:11.871181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:11.871181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:11.871181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 479/961] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:11.877260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:11.917626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f131e-16e5-45a9-bf8d-fe26c0dabb93 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:11.961379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:11.961991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:11.962395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:11.962696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:11.963469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:11.965106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:11.968717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING keystone.server.flask.common [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 11 10:44:11.998785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6dac1-2f85-474e-b56d-2c26e6b57fb2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.022110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.034737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.035813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.036297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.036839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.037734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:12.038262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.039869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.055845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 480/962] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:12.057648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.066211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 10:44:12.066211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 10:44:12.066211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [parameters: {'user_id': '78a00b9b6ed645cbb41926fa37baefd2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 10:44:12.066211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 11 10:44:12.082923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:44:12.083910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 10:44:12.085518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 10:44:12.086435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 10:44:12.087339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 10:44:12.095592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'user_id': '78a00b9b6ed645cbb41926fa37baefd2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:12.098035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 11 10:44:12.101298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 483/963] 10.222.0.48 () {60 vars in 1131 bytes} [Thu Aug 11 10:44:11 2022] POST /identity/v3/users => generated 160 bytes in 285 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:12.104294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.140110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.188079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb67c33-847a-49b2-a011-03ae56d84d37 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.224692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.225408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.225755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.225931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.233877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.235534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.239518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bae089-3b68-4919-9c3d-6c6fc8ce85cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:12.247728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:12.248731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:12.249742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 481/964] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:12.255872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.292565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.294538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.295024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.295495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.302121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.309189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.311831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:12.328643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:12.329684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 10:44:12.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.338589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 484/965] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:12.339801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.375590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b898-bd7e-4536-8323-8261a8d74815 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.411821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.412432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.412711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.413044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.413768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:12.414071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.415491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff4e37-c255-458c-bcac-5439abd732d5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 482/966] 10.222.0.48 () {56 vars in 1149 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb2186-4b12-4e98-83db-37dd5f02a3c9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb2186-4b12-4e98-83db-37dd5f02a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb2186-4b12-4e98-83db-37dd5f02a3c9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 483/967] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:12.439815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb2186-4b12-4e98-83db-37dd5f02a3c9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.440896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.451500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c90ee2-f883-499d-a0d4-8d5e14cbb76f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.487108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.487779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.488119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.488368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.489051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:12.489348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.490757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944f840-84b8-419c-a4ee-f7d1b89b64bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.535547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 485/968] 10.222.0.48 () {56 vars in 1101 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:12.535547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.535547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.535547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.548842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.553810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb2186-4b12-4e98-83db-37dd5f02a3c9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.554516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:12.556247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:12.606514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.606514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.606514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.606514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.668285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 486/969] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:12.682981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.684492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e965-c34d-474a-82ee-063784e995e8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.688766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.788664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.793282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:12.796674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 484/970] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:12.811892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f3cd-0861-4db1-bf1e-f66211a93628 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.811892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f3cd-0861-4db1-bf1e-f66211a93628 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.811892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f3cd-0861-4db1-bf1e-f66211a93628 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.811892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 485/971] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:12.811892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901f3cd-0861-4db1-bf1e-f66211a93628 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.821471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c2635-466f-4ff5-92ab-79f862e5f47a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.855113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.855738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.856091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.856369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.856952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.858326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-46995a1e-cc3a-426b-821e-bb96fd419079 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:12.867301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:12.868439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:12.869311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 487/972] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:12.870178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.870178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.870302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.887810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:12.889008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:12.907139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901f3cd-0861-4db1-bf1e-f66211a93628 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:12.940328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:12.941007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:12.941284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:12.941549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:12.951687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:12.953153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:12.954979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 488/973] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:12.955808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:12.968188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:12.969187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:12.970095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:12.970095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:12.970095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:12.970095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 486/974] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:12.975165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.026616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.027126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.055034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110bd32-6440-47e3-926b-b1d040129ed7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.071600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa31163-85d1-4201-b809-cc74eed494c6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.088306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.088901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.089173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.089439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.090052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:13.090600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.091974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.096550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 489/975] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:13.097288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.105519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.106113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.106385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.106650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.107283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:13.107575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.108950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.121466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 487/976] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:13.122425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.196239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdb29f-24f4-45bd-8c8a-af291425cad6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.220375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131ec5f-fa3b-403e-85b6-0a55958b9d26 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.232325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.232914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.233196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.233337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.233776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.234081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.235411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.239466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.253434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.254031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.254308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.254576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.257026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:13.257026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.257590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.262132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 488/977] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:13.262888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.311566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 490/978] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:13 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:13.312429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.366889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973d2d8-df39-46b5-8b95-cbf5ad1b2853 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.411192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.411865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.412144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.412416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.420887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.422693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.425297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfd607-da17-430b-9734-c5106e3ae491 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:13.435495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:13.436660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:13.437801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:13.439580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 489/979] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:13.446568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.471420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.472113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.472545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.472938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.473602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.475776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a10d1-3ed8-4e76-b09d-b350b332e503 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.480640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.550400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be896d6b-5b29-405a-9fd7-03376fb2ef0a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.583362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.583921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.584241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.584591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.585809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:13.586108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.587527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.605920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 490/980] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:13.606217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 491/981] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:13.612261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:13.714572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:13.724517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c57ae-ac84-44a3-811c-4d465ee4e10e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:13.768851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:13.769488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:13.769752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:13.770021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:13.770744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] RBAC: Adding query filter params (role.id=735e389132614971ad154ae4a9c80a7c, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=8bf57a090d4f4ce5b950ccb1c1c584a6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:13.771051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:13.773029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cf27b-c016-472c-bf8c-d104d3c0b4e4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:13.779113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 491/982] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:13 2022] GET /identity/v3/role_assignments?user.id=8bf57a090d4f4ce5b950ccb1c1c584a6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=735e389132614971ad154ae4a9c80a7c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:15.329270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1352-0278-4e67-81dd-0c9af54917ed None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.329800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1352-0278-4e67-81dd-0c9af54917ed None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.329800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1352-0278-4e67-81dd-0c9af54917ed None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.331280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 492/983] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:15.336775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.337185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.337355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.355314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:15.356512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:15.424153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 492/984] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:15.446397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1352-0278-4e67-81dd-0c9af54917ed None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:15.499656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:15.547060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c1352-0278-4e67-81dd-0c9af54917ed None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:15.581852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:15.582408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.582685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.582954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.583530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:15.587272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7bb3129d-4f9d-48a5-9be8-5ab023a9d4e3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:15.596126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:15.597131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:15.605549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 493/985] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:15.609713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:15.714577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:15.732695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0904a6c-71ed-4183-881e-0a42ea2b3543 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:15.746238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f945-14df-46fe-a894-ff9da8b29ed4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.746528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f945-14df-46fe-a894-ff9da8b29ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.746802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f945-14df-46fe-a894-ff9da8b29ed4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.748315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 494/986] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:15.752938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47c4d-30f4-4eda-a1f3-db55fc6cf54d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.753225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47c4d-30f4-4eda-a1f3-db55fc6cf54d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.753503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd47c4d-30f4-4eda-a1f3-db55fc6cf54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.769214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:15.769812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.770199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.770473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.771092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:15.771391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:15.771720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9dd47c4d-30f4-4eda-a1f3-db55fc6cf54d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:15.773324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:15.773506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9dd47c4d-30f4-4eda-a1f3-db55fc6cf54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:15.778239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 493/987] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:15.783685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:15.842361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 495/988] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:15.843668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75efa92-fa4b-4893-9be7-31b5e92663cc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.843980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75efa92-fa4b-4893-9be7-31b5e92663cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.844251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75efa92-fa4b-4893-9be7-31b5e92663cc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.845605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 496/989] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:15.850637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.850920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.851189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.870333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:15.871503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:15.890361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391acd2-4795-4004-96c1-a9c00ebdb66f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:15.913428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:15.923649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:15.924278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:15.924548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:15.924812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:15.928707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:15.928707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:15.928707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:15.932644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 494/990] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:15.933542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:15.942847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 497/991] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:15.943792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:15.949857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.037226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52154878-69f7-40b7-b8ae-5ede00cd795d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.040033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc329c7-4a7c-43cb-9137-c4c57053a2c2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.070114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.070735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.071007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.071275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.071872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.073219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.073338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.073841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.074119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.074389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.075075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] RBAC: Adding query filter params (interface=public, service_id=f61a2e91c1b84df3996a4de373d7ebcb, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.075385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.076741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.081613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 498/992] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:44:15 2022] GET /identity/v3/endpoints?service_id=f61a2e91c1b84df3996a4de373d7ebcb&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:16.081772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:16.082746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:16.083608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:16.084934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.085007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 495/993] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:16.087733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.128209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.159782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.183862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e820a63-6689-4e10-894e-b2f119a9043a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.184739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614d3d1-8d1a-4304-85d1-1437c235960a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.216589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.217182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.217457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.217723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.218239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.218353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.218855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.219129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.219395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.219647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.220016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.221335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.226262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 496/994] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services => generated 1727 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:44:16.227041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:16.230256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:16.231552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:16.232452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 499/995] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:16.237705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.346801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e909d39-00b7-47b5-968d-1bf6815a5325 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.358084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb14e-a4d3-4adb-b8de-706fc401592c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.361596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.387318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.387877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.388151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.388480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.389108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.389424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.390786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.399437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.399910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 497/996] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:16.400403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.400687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.401024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.401661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.401965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.403310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.405405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.413616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 500/997] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:16.414799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.415744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.512763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffa777-4c58-4d9f-a337-05252746dcc9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.525364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9066-fe49-4576-9afb-e2ea2ad5bbcf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.547513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.548177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.548426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.548702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.549329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.549630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.551077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.560816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 498/998] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:16.561966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.566922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.567522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.567825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.568205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.569294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.569591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.570915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.583806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 501/999] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:16.645373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.683798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fe69-74f3-4727-a843-bfe912e38a00 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.721455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.722073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.722351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.722623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.723332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] RBAC: Adding query filter params (interface=admin, service_id=b8f0895d2e6a4a25a5ab47e63ce1253a, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:16.723641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.725687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befadc43-6074-476f-93eb-8776a51e4380 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.731722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 499/1000] 10.222.0.48 () {56 vars in 1249 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/endpoints?service_id=b8f0895d2e6a4a25a5ab47e63ce1253a&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:16.737535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:16.770277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:16.839337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841ed93-387c-4e87-beca-20bf52943dbf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:16.873465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:16.874067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:16.874347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:16.874618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:16.875217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:16.876620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f163c7f-bc42-4e50-8390-c0e2a636b6b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:16.883113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 502/1001] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:16 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:44:18.249837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951349-72ec-4974-b221-605372aaf177 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.250402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951349-72ec-4974-b221-605372aaf177 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.250402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951349-72ec-4974-b221-605372aaf177 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.251795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 500/1002] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:18.257065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.257490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.257609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.276707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:18.277913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:18.347645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 503/1003] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:18.368191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc951349-72ec-4974-b221-605372aaf177 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:18.480842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc951349-72ec-4974-b221-605372aaf177 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:18.515904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:18.524809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:18.534114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.534114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.534114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.534114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:18.534114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-69b41f4c-605e-43cd-8262-7c6700812036 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:18.536752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:18.538302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:18.553858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 501/1004] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:18.554699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:18.584674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:18.596500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13e9b-1040-4e1c-b39d-4f6cfd1ecde5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.596836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13e9b-1040-4e1c-b39d-4f6cfd1ecde5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.597060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff13e9b-1040-4e1c-b39d-4f6cfd1ecde5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.599294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 502/1005] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:18.607567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.607882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.608163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.627263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:18.628492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:18.671370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1c19d-82f3-4de0-a933-7b5ac9f96b9f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:18.702413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 503/1006] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:18.708373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:18.708990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.709285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.709552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.710169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:18.710467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:18.712936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:18.722140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 504/1007] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:18.723081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:18.727538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:18.825307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:18.826977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2360cca-678a-4e32-bc5d-ad750624c5d2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:18.835331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cda9f-140c-46ae-9126-e79a3e88228c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:18.851455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:18.862799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:18.863408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.863717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.864013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.864641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:18.864940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:18.866853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:18.876297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 504/1008] 10.222.0.48 () {56 vars in 1112 bytes} [Thu Aug 11 10:44:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:18.877268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:18.878100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:18.878702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.878976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.879238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.880027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:18.882049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32753c-69fb-4e9d-99ea-c7f35388db22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:18.978831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 505/1009] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:18 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:18.979026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ba018-304c-420a-aa89-fda65196792c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:18.980440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea008-4c9d-483e-953f-57635114668f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.980718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea008-4c9d-483e-953f-57635114668f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.980989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea008-4c9d-483e-953f-57635114668f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:18.982381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 506/1010] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:18.988522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:18.988522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:18.989205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.009666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:19.010880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:19.027815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:19.028435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:19.028712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:19.028982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.029653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:19.031067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:19.084644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 507/1011] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:19.115797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 505/1012] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:19.116091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:19.206265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:19.226722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e33bd-242e-49d8-ba1c-e1b6cac5ae6d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:19.270694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:19.271345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:19.271645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:19.271947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.272530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:19.273879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:19.283202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:19.284362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:19.285229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 508/1013] 10.222.0.48 () {56 vars in 1126 bytes} [Thu Aug 11 10:44:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:19.290864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:19.336752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:19.400638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ed98b-926a-404e-b93d-15827374061e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:19.434773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:19.435386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:19.435658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:19.435956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.436577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:19.436874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:19.438293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:19.447792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 506/1014] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:19.449650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:19.550961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f9777-23a9-4742-985d-fa64cc963b4d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:19.587866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:19.587866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:19.587866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:19.587866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.588243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:19.588243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:19.588581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28015-8273-43d7-8ad9-9d008839c50f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:19.594599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 509/1015] 10.222.0.48 () {56 vars in 1122 bytes} [Thu Aug 11 10:44:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:19.602125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:19.706649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562c3a2-5d98-4b68-846d-80370f139e33 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:19.709764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:19.740003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:19.741128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:19.741406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:19.741662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:19.742316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:19.743658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5ec22-cf8f-4492-af7a-1f6f4555c077 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:19.832674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 507/1016] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:21.006670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d67-e128-4e5c-963d-142637749530 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.007198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d67-e128-4e5c-963d-142637749530 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.007198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d67-e128-4e5c-963d-142637749530 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.009674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 510/1017] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:21.014049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a651b-4025-406e-850c-ceed3a6b38ce None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.014454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a651b-4025-406e-850c-ceed3a6b38ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.014613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a651b-4025-406e-850c-ceed3a6b38ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.032849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7d3a651b-4025-406e-850c-ceed3a6b38ce None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:21.034008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d3a651b-4025-406e-850c-ceed3a6b38ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:21.105267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 508/1018] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:21.127752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76d67-e128-4e5c-963d-142637749530 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.132137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.225789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c57bc-3a22-42df-af55-351821695245 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.226187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c57bc-3a22-42df-af55-351821695245 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.226325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c57bc-3a22-42df-af55-351821695245 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.227434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76d67-e128-4e5c-963d-142637749530 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.227954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 509/1019] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:21.232629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.232915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.233188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.250976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:21.252196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:21.262530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.263114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.263433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.263702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.264306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.265653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:21.275098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:21.276228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:21.277244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 511/1020] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:21.283181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.334239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 510/1021] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:21.353930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.357136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.378788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.392272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7a181-36b9-4112-8b1c-f6050e049f1b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.425632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.426237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.426510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.426779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.427400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:21.428241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.429716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.434702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 512/1022] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:21.440341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.464920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1bfeb-adb6-4fe1-bf44-390e91d6879a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.501772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.502394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.502673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.502940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.513359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:21.517901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:21.518857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:21.519785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 511/1023] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:21.525020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.551487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ec7f9-37bf-4dfc-83e7-d6bf42340b99 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.581390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.587128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.588109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.588653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.588931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.589570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:21.590274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.591545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.597809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 513/1024] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:21.608719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.634725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.656556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d3dce-ac30-43b3-a6cf-8c8cf23fc8c7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.694274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.694880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.695153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.695482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.696177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:21.696477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.697840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.706484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 512/1025] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:21.712462 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.726999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e857e-4915-41e9-8657-5ab90fd20440 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.766186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.766876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.767159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.767433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.768167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] RBAC: Adding query filter params (interface=public, service_id=5674bf28975f454292807e4c48ed9a99, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:21.768477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.769826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.779311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 514/1026] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/endpoints?service_id=5674bf28975f454292807e4c48ed9a99&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:21.780344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.826146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4b104-9db4-4ea6-91ad-94d69ba07be7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.844992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.861239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.861868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.862143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.862412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.873615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.875087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.877077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ac1ec6e3-efce-457c-85a2-238532cdd92a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:21.889749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:21.890869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:21.891841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:21.891841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:21.891841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:21.891841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 513/1027] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:21.892098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46b0f7-a1f0-4138-a3ef-234813522b24 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:21.892396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00601-8493-4dd5-b910-a7bf7459036b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.892675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00601-8493-4dd5-b910-a7bf7459036b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.892944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd00601-8493-4dd5-b910-a7bf7459036b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.894736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 514/1028] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:21.896575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd00601-8493-4dd5-b910-a7bf7459036b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:21.925020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:21.925613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.925894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.926164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.926764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:21.928737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfcd39-8b92-4e62-9755-14aae353f2d7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:21.939890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 515/1029] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:44:21.941827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83198-affc-4f65-983c-a4673181e12e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:21.942115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83198-affc-4f65-983c-a4673181e12e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:21.942388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83198-affc-4f65-983c-a4673181e12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:21.965605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69e83198-affc-4f65-983c-a4673181e12e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:21.966855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69e83198-affc-4f65-983c-a4673181e12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:22.023973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd00601-8493-4dd5-b910-a7bf7459036b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.049472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 516/1030] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:22.067584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.068959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.069308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.069645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.070370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:22.070739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.072405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.073402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e83198-affc-4f65-983c-a4673181e12e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.085446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 515/1031] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:22.091894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.138031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:22.148337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:22.182442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e83198-affc-4f65-983c-a4673181e12e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.197811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa182a9-4a07-43a5-90ce-22965d363f0d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.216461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.217084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.217358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.217623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.218724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.220166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.222037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 517/1032] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:22.228177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.239228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.239933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.240252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.240525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.248355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.249802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:22.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:22.263324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:22.265025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 516/1033] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:22.271013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.340702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea378f-767a-4137-98a9-2d555f7da0db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.364580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:22.376292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.379942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.379942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.379942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.379942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.379942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.384033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee50ced-df03-447d-b1c8-9ad5724841aa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.399257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:22.419100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.419703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.420019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.420291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.421010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:22.421313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.422733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.439414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 517/1034] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:22.445500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.471790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 518/1035] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:22 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:22.567372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef567b-fa4f-4b68-bab2-71b12c17e941 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.614621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.615805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.616095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.616369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.661645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:22.689444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.690897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.795793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 518/1036] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 10:44:22.797724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:22.898393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90706e0-0e4a-44ff-8e07-d2088fa37150 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:22.931374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:22.932069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:22.932359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:22.932629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:22.943484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:22.945423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:22.959390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:22.960565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:22.961484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:22.961484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:22.961484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:22.961484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 519/1037] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:22.967565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:23.039590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:23.071911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577f9d-624b-4e5a-85bb-12b0396e3c67 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:23.106318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:23.107036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.107431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.107818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:23.108548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:23.108943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:23.110391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:23.122251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 519/1038] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:23.127592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:23.155125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:23.226688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f27cb-a273-4729-a0ae-4e177e3e0368 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:23.259679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:23.260491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.260863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.261226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:23.267748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:23.269273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:23.280133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:23.281197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:23.282356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:23.283846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 520/1039] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:23.289860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:23.387342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd1a64-1a81-427a-a95c-64116026ba7f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:23.416261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:23.421588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:23.422249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.422588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.422924 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:23.423669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:23.424059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:23.425530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda9658-0a9c-45e1-a8f9-b39c6a57a9f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:23.437209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 520/1040] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:23.442950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:23.527264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:23.542953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8f004-7aa2-49c5-af38-fc6c3f3b997d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:23.576816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:23.577415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.577694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.577960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:23.578720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] RBAC: Adding query filter params (role.id=735e389132614971ad154ae4a9c80a7c, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:23.579130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:23.580494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5b2ed-163a-4a5a-8ee6-853ccd3dd0bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:23.589933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 521/1041] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:23 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=735e389132614971ad154ae4a9c80a7c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:23.975496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a104d-2965-473c-9968-9a4b2869effc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.975944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a104d-2965-473c-9968-9a4b2869effc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.976122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a104d-2965-473c-9968-9a4b2869effc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:23.977574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 521/1042] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:23.982652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:23.982946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:23.983223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.001504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:24.002686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:24.072799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 522/1043] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:24.095312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a104d-2965-473c-9968-9a4b2869effc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.197183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a104d-2965-473c-9968-9a4b2869effc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.231975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.232615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.232964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.233304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.233947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.235392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:24.244695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:24.246014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 10:44:24.247093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:24.248233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 522/1044] 10.222.0.48 () {56 vars in 1105 bytes} [Thu Aug 11 10:44:24 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:24.253667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.333118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:24.353193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fabe69-f559-4c52-b205-2632de3fb0d2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.386467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.387064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.387342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.387613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.388266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:24.388566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.389908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c94fc8-8de9-41d3-b1ac-b93f01546f1c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.399694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 523/1045] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:24.401464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.457674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:24.503275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904a05-702b-45b4-9951-4f74682baf89 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.503444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5794dd3-bb6a-43ca-803d-f4923ece8af2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.503732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5794dd3-bb6a-43ca-803d-f4923ece8af2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.504033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5794dd3-bb6a-43ca-803d-f4923ece8af2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.505522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 524/1046] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:24.510233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.510516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.510789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.528896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:24.530070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:24.541578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.542186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.542464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.542740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.543359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:24.543677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.545033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.550186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 523/1047] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:24.556018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.607801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 525/1048] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:24.631894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.672422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec674aa-4703-4d25-b7d4-24bd929c2e67 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.694457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:24.708864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.709687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.710226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.710406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.711027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.712419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f753e6-67e2-4a5e-9b31-a339f8f4bc83 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.718717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:24.744815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167817b-0c0d-437d-9f41-0942af121dab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.777188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.777791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.778067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.778339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.779486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.780935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.782503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 526/1049] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:24.788659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:24.823796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 524/1050] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:44:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:24.906159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94137f-e9e2-481d-9e66-d94f182575f7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:24.949151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:24.949762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:24.950042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:24.950316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:24.950959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:24.952431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c8639-9f34-4af6-84f8-367f4bade05c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:24.963254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:25.077705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 527/1051] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:44:24 2022] POST /identity/v3/users => generated 285 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:44:25.557554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093bd5-5a1f-4eef-83a5-570bc65010da None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:25.558087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093bd5-5a1f-4eef-83a5-570bc65010da None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:25.558087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093bd5-5a1f-4eef-83a5-570bc65010da None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:25.559441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 525/1052] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:25.564866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:25.565314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:25.565392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:25.587503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:25.587503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:25.659210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 528/1053] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:25.681126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b093bd5-5a1f-4eef-83a5-570bc65010da None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:25.780800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b093bd5-5a1f-4eef-83a5-570bc65010da None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:25.813703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:25.814245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:25.814515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:25.814779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:25.825925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:25.830510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:25.831456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:25.832358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:25.836074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 526/1054] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:25.837665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:25.940196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d15c8-b835-4203-aa91-2610e36fbc7b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:25.965800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:25.977497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:25.978167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:25.978498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:25.982899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:25.982899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:25.982899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:25.982899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:25.994185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 529/1055] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:25.995436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.033384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:26.097692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e74b3-ee8e-4638-a123-9338993dae29 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.131296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.131939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.132216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.132481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.142655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.144103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:26.157639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:26.158597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:26.164684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:26.164684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:26.164684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:26.164684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 527/1056] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:26.165003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.264300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a024445-b486-4bf6-85a9-c4c873b387cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.297686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.298282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.298563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.298825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.299451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:26.299745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.301101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.313013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 530/1057] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:26.319056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.405882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:26.418675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a822f3c-d7e1-4aa8-8057-2361a5ac1286 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.451680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.452321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.452598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.452866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.460690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.462109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:26.473024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:26.474006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:26.475526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:26.479284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 528/1058] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:26.480642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.526424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:26.578770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8970be5-d395-4581-b904-193e06e8d323 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.619429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.620055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.620330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.620597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.621302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:26.621599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.622994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.643051 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 531/1059] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:26.649466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.757135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48b702-c0ba-4bfd-b918-325f7760f39f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.790406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.791030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.791298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.791562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.803371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.804863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.812691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2b7406d2-202c-4f39-a736-c17bc3c63e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:26.819383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:26.820706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:26.821838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:26.821838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:26.821838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:26.823923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 529/1060] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:26.827801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.834429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1a1af-60ea-45a9-90e8-514994c61cc8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.834735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1a1af-60ea-45a9-90e8-514994c61cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.835590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1a1af-60ea-45a9-90e8-514994c61cc8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.836678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 530/1061] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:26.841809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.842085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.842353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.861337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:26.862513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:26.933362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 531/1062] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:26.943931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f754-6143-4e19-91df-85ffa96d19b0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:26.943931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:26.957075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:26.969394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:26.970061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:26.970371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:26.970605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:26.971287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:26.971591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:26.973308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:26.991020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 532/1063] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:26.992141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.075789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b1b6d-e491-45de-8a21-3c643b2d27b8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.107336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.109784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c430e8d-3908-480c-bda8-aaf182fbaf62 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.113987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.114693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.114906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.118952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.143298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.143950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.144229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.144500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.151461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.152910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5e4c97a0-6b00-439d-ba58-3f60a113d8dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:27.164327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:27.165329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:27.166232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 533/1064] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:27.166898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24985abc-427e-45d8-914d-2255cc461352 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.167140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24985abc-427e-45d8-914d-2255cc461352 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.167413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24985abc-427e-45d8-914d-2255cc461352 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.168616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 534/1065] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:27.171959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24985abc-427e-45d8-914d-2255cc461352 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.190287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.191726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c30ca-b4b2-48dd-aa2a-d92d8890470f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.197176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.273071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24985abc-427e-45d8-914d-2255cc461352 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.287585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 532/1066] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:27.288849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.289130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.289401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.306461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:27.306644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.307243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.307518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.307626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:27.307807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.308488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:27.308793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.310201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.321417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 535/1067] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:27.322190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.330416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.374419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 533/1068] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:27.375643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.406811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.421785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e84d-0ad1-4130-98ef-6e3daa8ccfb1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.455359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.455997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.456270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.456539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.467421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.468880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.472486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272f1d0-ea6d-4166-8726-1080494757f2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:27.483430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:27.484593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:27.485545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:27.485545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:27.485545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:27.485545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 536/1069] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:27.485545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.511301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.511983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.512266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.512541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/735e389132614971ad154ae4a9c80a7c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.514701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=d622a3b3cc2a4e469033cb2d5c66d2ed, role_id=735e389132614971ad154ae4a9c80a7c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.516164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.581114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fad6d-a0ba-42c1-8bbe-0b17df348eb6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.590951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 534/1070] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:44:27 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/d622a3b3cc2a4e469033cb2d5c66d2ed/roles/735e389132614971ad154ae4a9c80a7c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:44:27.591792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.616014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.631031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.631675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.631986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.632255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.635953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.645344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:27.650605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:27.651892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:27.653721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 537/1071] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:27.659611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.707918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5debcd1-21c7-466b-828a-dd7bc28da044 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.752551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.753171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.753495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.753780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.754602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:27.755336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.757109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.770561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 535/1072] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:27.774211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb89f2-74d8-48cb-906c-95ee389b12ec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.815598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.816249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.816528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.816798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:27.817473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:27.817824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:27.819230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:27.835534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 538/1073] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:27.837714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:27.930674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:27.949840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845718-3684-4f49-905c-54c1f040acdf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:27.987807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:27.988440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:27.988710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:27.988979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] PATH_INFO: `/v3/users/b69833240e1445b19c259a812011ad9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:28.003266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] RBAC: Authorizing `identity:get_user(user_id=b69833240e1445b19c259a812011ad9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:28.004731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:28.006954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 536/1074] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:27 2022] GET /identity/v3/users/b69833240e1445b19c259a812011ad9a => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:28.012987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:28.076728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:28.118376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479aff5-49e8-498d-952d-28db6c5bf898 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:28.152482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:28.153077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:28.153355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:28.153626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:28.219646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:28.221117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:28.275340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:28.315700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 539/1075] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:28.322096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:28.419809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22b43-1a91-4848-b291-4ecfbb25924f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:28.452734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:28.453351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:28.453630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:28.453902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:28.455004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:28.456463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2920fd-9a2b-4434-b6a1-4db8503dd501 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:28.458085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 537/1076] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:28 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:28.464180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:28.559214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df910123-8177-4245-80b8-ad7e5dcc686e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:28.592327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:28.592936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:28.593184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:28.593451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:28.594163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652, user.id=b69833240e1445b19c259a812011ad9a) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:28.594466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:28.595883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d48f64-8fda-4f71-8e6f-d4e776244a3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:28.611787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 540/1077] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:28 2022] GET /identity/v3/role_assignments?user.id=b69833240e1445b19c259a812011ad9a&scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:29.581927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c5b5-9e2e-41a3-979d-8b5ac68394a4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:29.582474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c5b5-9e2e-41a3-979d-8b5ac68394a4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:29.582562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5c5b5-9e2e-41a3-979d-8b5ac68394a4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:29.587062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 538/1078] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:29.594163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:29.594583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:29.594713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:29.612842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:29.614017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:29.630073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:29.699561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 541/1079] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:29.722381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c5b5-9e2e-41a3-979d-8b5ac68394a4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:29.731553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:29.819662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5c5b5-9e2e-41a3-979d-8b5ac68394a4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:29.852238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:29.852791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:29.853069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:29.853342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:29.863155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:29.868338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:29.869260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:29.870199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 539/1080] 10.222.0.48 () {56 vars in 1120 bytes} [Thu Aug 11 10:44:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:29.875343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:29.971232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eff02-5e39-41a3-8f89-a0b0ab22d848 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.004226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.004843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.005116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.005381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.006030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:30.006326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.007668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.015360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 542/1081] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:44:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:30.021032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.090995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.124825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a62550-3394-43c5-8fbc-63bbab504761 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.157930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.158546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.158821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.159090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.169256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.170680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:30.185146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:30.186110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:30.187058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:30.187058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:30.187058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:30.187911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 540/1082] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:30.192026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.208888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.291107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707f77f-d2a7-4472-afc1-b65ec23fb73a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.324151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.324744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.325020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.325287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.325919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:30.326220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.327565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.338843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 543/1083] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:30.344854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.446403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6cb50-0346-4618-8b6f-602587de4f3a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.463018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.480099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8472ef8a-9d7e-45fd-af8d-9db275f3c5ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:30.501506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:30.502093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:30.503201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:30.504181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 541/1084] 10.222.0.48 () {56 vars in 1093 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:30.507951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.596693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2c0a-d761-406f-b81c-73520f23a5ca None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.597214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2c0a-d761-406f-b81c-73520f23a5ca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.597214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2c0a-d761-406f-b81c-73520f23a5ca None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.598617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 542/1085] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:44:30.603417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.603698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.604004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.621675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:30.622849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:30.646437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.669875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2cc91-a2d2-4633-8851-fa16992f7b9a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.703455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.703775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 543/1086] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:30.704163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.704506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.704844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.705624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:30.705991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.707471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.730385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 544/1087] 10.222.0.48 () {56 vars in 1133 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:30.730569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.731342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.824449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e302e-6a5c-4b9e-a5bd-94350cddc99f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.825996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cd9e8-e1fd-48a5-b287-a2d84f91555f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:30.833880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.854852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:30.858830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.859450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.859806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:30.859993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.860272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.860367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:30.860611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:30.860883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:30.871266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:30.875975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:30.876906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:30.877766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 544/1088] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:30.883565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:30.931196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:30.932659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:30.981213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41507d0e-2dda-41ed-83a3-8426c26438dd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.013636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.014234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.014512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.014788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.015443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:31.015750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.017140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.026269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 545/1089] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:31.031797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 545/1090] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 10:44:31.035180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.035316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.063703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.064184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.132333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d0242-ff9a-41dd-a670-078c8421c937 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.133449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83b882-1c3e-41ba-813f-3127a9f707f6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.165716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.166321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.166592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.166862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] PATH_INFO: `/v3/users/b69833240e1445b19c259a812011ad9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.167412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.167987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] RBAC: Authorizing `identity:get_user(user_id=b69833240e1445b19c259a812011ad9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.168106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.168326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.168597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.169402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.171133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 546/1091] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/users/b69833240e1445b19c259a812011ad9a => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:31.177026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.179294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.180733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:44:31.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:31.196138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:44:31.197066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:44:31.197066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:44:31.197066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:31.197066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 546/1092] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:31.201581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.271646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.275279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588a7f3-dabd-4992-9efd-871937d60791 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.282854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.298245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a50d31-23f7-4bfc-9915-91608b8bd040 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.308232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.308851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.309128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.309402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.310500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.311956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.313580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 547/1093] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:31.319806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.331875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.332475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.332750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.333547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.334178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:31.334498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.335858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.347102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 547/1094] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:31.352563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.418854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1fa89-f639-40e6-a023-1764278e664c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.449041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aa338-5eec-44e1-af93-6ce597002b6d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.451958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.452599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.452876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.453147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652/users/b69833240e1445b19c259a812011ad9a/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.455021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] RBAC: Authorizing `identity:create_grant(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652, user_id=b69833240e1445b19c259a812011ad9a, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.456464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.481766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.482380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.482661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.482933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.489426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.490829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:44:31.501641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:31.505440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:31.506566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 548/1095] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:44:31.508846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.512065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.574213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 548/1096] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:44:31 2022] PUT /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652/users/b69833240e1445b19c259a812011ad9a/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:44:31.605701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122621a-f8f1-4ca7-b172-b57945cb9750 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.643212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.643847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.644128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.644400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.645091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:31.645433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.646942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e6bcd-c5f4-40e6-95bd-290d1345411a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.660299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 549/1097] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:31.666464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:31.695891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:31.769868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb3f2a-67ec-48a9-a4c2-965c54d6a730 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:31.803780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:31.804383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:31.804662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:31.804934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:31.805656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] RBAC: Adding query filter params (role.id=735e389132614971ad154ae4a9c80a7c, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=d622a3b3cc2a4e469033cb2d5c66d2ed) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:31.805964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:31.807384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c2e1-6129-404b-a6e2-61f0368d600c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:31.813095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 549/1098] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:31 2022] GET /identity/v3/role_assignments?user.id=d622a3b3cc2a4e469033cb2d5c66d2ed&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=735e389132614971ad154ae4a9c80a7c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:33.554272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4150db-4f95-4917-a15b-246e6f01a30d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:33.554808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4150db-4f95-4917-a15b-246e6f01a30d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:33.554808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4150db-4f95-4917-a15b-246e6f01a30d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:33.556203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 550/1099] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:33.561283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:33.561682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:33.561823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:33.579442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:33.580628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:33.645907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 550/1100] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:33.668764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4150db-4f95-4917-a15b-246e6f01a30d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:33.729792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:33.765762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4150db-4f95-4917-a15b-246e6f01a30d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:33.798959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:33.799561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:33.799868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:33.800142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:33.810431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:33.814999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:33.816112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:33.817144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 551/1101] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:33.821965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:33.842768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:33.917459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee42fb-8998-4066-b90d-886ae7bc6bff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:33.949847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:33.950436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:33.950710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:33.950977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:33.951624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:33.951950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:33.953409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:33.962200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 551/1102] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:33.968240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:34.064434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810c981-d24a-493d-83e8-2a87d751948f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:34.086119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:34.096320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:34.097449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:34.097449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:34.097635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] PATH_INFO: `/v3/users/b69833240e1445b19c259a812011ad9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:34.098687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] RBAC: Authorizing `identity:get_user(user_id=b69833240e1445b19c259a812011ad9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:34.100149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:34.101681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 552/1103] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:33 2022] GET /identity/v3/users/b69833240e1445b19c259a812011ad9a => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:34.107253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:34.186569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:34.204044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276c6a1-e5a6-407e-964f-983a4acf349d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:34.236870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:34.237445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:34.237717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:34.237981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:34.301917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:34.303340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:34.378207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:34.395410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 552/1104] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:34.401248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:34.496524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09092a-421e-4806-b28b-f51859365878 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:34.527535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:34.528165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:34.528442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:34.528708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:34.529792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:34.531181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d36f31-7971-472c-9bb1-3f02be05a97f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:34.532787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 553/1105] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:34 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:34.538962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:34.634699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcd7728-057f-4a03-80f7-fabcf07dd7e8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:34.667468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:34.668082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:34.668359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:34.668626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:34.669336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652, user.id=b69833240e1445b19c259a812011ad9a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:34.669640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:34.671042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdb546-2248-4c36-bffa-96851cf702f7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:34.676589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 553/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:34 2022] GET /identity/v3/role_assignments?user.id=b69833240e1445b19c259a812011ad9a&scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:36.649083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c0de-9b40-487e-9be4-61725981c177 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:36.649565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c0de-9b40-487e-9be4-61725981c177 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:36.649658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c0de-9b40-487e-9be4-61725981c177 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:36.650991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 554/1107] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:36.656081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:36.656492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:36.656621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:36.674571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:36.675737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:36.716258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:36.740460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 554/1108] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:36.763990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5c0de-9b40-487e-9be4-61725981c177 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:36.828349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:36.870434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5c0de-9b40-487e-9be4-61725981c177 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:36.902832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:36.903443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:36.903721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:36.904019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:36.905113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:36.906519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41f85e-fe45-40ac-98e0-c3078e9d19fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:36.908107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 555/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:36.914620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:37.008585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c8088-da0a-4bdd-971d-010a8eb0074b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:37.041391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:37.041975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:37.042253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:37.042525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:37.043170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:37.044600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdddf4-ee6e-48f6-8c22-c70c25591f5f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:37.127160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 555/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:44:36 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:39.151495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe577cf-28bd-4456-93c9-698c3c2cb856 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.152124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe577cf-28bd-4456-93c9-698c3c2cb856 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.152124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe577cf-28bd-4456-93c9-698c3c2cb856 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.153400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 556/1111] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:39.158482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f493b-8956-45c5-82ad-966879728230 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.158930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f493b-8956-45c5-82ad-966879728230 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.159030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f493b-8956-45c5-82ad-966879728230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.176978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-673f493b-8956-45c5-82ad-966879728230 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:39.178127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-673f493b-8956-45c5-82ad-966879728230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:39.202470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:39.241943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 556/1112] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:39.264325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe577cf-28bd-4456-93c9-698c3c2cb856 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:39.361608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe577cf-28bd-4456-93c9-698c3c2cb856 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:39.386125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:39.393436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:39.394041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.394321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.395110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] PATH_INFO: `/v3/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.396527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] RBAC: Authorizing `identity:get_role(role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:39.397861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:39.399138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 557/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:39 2022] GET /identity/v3/roles/e0028b79d128480ba32f23616eb613c1 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:39.404842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f493b-8956-45c5-82ad-966879728230 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:39.499973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f493b-8956-45c5-82ad-966879728230 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:39.532733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:39.533340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.533615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.533883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] PATH_INFO: `/v3/users/8d919376a0624c4891db0ee5589d27e1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.546196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] RBAC: Authorizing `identity:get_user(user_id=8d919376a0624c4891db0ee5589d27e1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:39.547618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:39.549211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 557/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:39 2022] GET /identity/v3/users/8d919376a0624c4891db0ee5589d27e1 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:39.555683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:39.650925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4a023-1c95-4059-bed1-318f8b24de76 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:39.682747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:39.683275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.683547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.683838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.742322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:39.748389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:39.755534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:39.841859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 558/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:39.848243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:39.869055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:39.943096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ccc55-ae9f-4fbf-9446-e5fdde0f435d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:39.982313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:39.982933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:39.983210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:39.983482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:39.984620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:39.986043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c185250-7456-4dfa-8c6c-4bc8559a9a1f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:39.987681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 558/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:39 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:39.993791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:40.081277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:40.087162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361d59-d034-4dee-8d5f-709d6c7c941c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:40.118949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:40.119526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:40.119819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:40.120089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:40.120804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652, user.id=8d919376a0624c4891db0ee5589d27e1) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:40.121357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:40.123028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d9fe-4141-4133-b789-ec37015d551f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:40.128311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 559/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:39 2022] GET /identity/v3/role_assignments?user.id=8d919376a0624c4891db0ee5589d27e1&scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652&role.id=e0028b79d128480ba32f23616eb613c1 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:42.064200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47b0f8-3afb-4d63-afc9-7a2b915196a7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.064737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47b0f8-3afb-4d63-afc9-7a2b915196a7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.064737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47b0f8-3afb-4d63-afc9-7a2b915196a7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.066570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 559/1118] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:42.071898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.072325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.072444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.090602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:42.091753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:42.154262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 560/1119] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:44:42.176219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47b0f8-3afb-4d63-afc9-7a2b915196a7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:42.262540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:42.274763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47b0f8-3afb-4d63-afc9-7a2b915196a7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:42.307458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:42.308095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.308374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.308644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] PATH_INFO: `/v3/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.310005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] RBAC: Authorizing `identity:get_role(role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:42.311361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:42.312698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 560/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:42 2022] GET /identity/v3/roles/e0028b79d128480ba32f23616eb613c1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:42.319246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:42.404359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:42.413547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5da57a-167f-4545-bcb3-e8c082b84453 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:42.446610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:42.447216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.447494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.447786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] PATH_INFO: `/v3/users/8d919376a0624c4891db0ee5589d27e1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.448892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] RBAC: Authorizing `identity:get_user(user_id=8d919376a0624c4891db0ee5589d27e1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:42.450293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:42.451925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 561/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:42 2022] GET /identity/v3/users/8d919376a0624c4891db0ee5589d27e1 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:42.457432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:42.559524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622c1b-496b-4454-8c44-24f131e14b79 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:42.593443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:42.593995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.594270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.594538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.595628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:42.597081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69565cae-b4f0-4168-81b5-5aa988d54543 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:42.598743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 561/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:42 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:42.604719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:42.698075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799bd87-3c9d-4496-9803-dc646134df2f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:42.730149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:42.730774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:42.731046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:42.731311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652/users/8d919376a0624c4891db0ee5589d27e1/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:42.733414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652, user_id=8d919376a0624c4891db0ee5589d27e1, role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:42.734815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413368a-f367-40ee-b62a-d89081ac66f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:42.852026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 562/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:44:42 2022] PUT /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652/users/8d919376a0624c4891db0ee5589d27e1/roles/e0028b79d128480ba32f23616eb613c1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:44:44.800929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6dad-28bc-4a00-b79c-a329d066864f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:44.801472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6dad-28bc-4a00-b79c-a329d066864f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:44.801472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b6dad-28bc-4a00-b79c-a329d066864f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:44.802811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 562/1124] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:44.808002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:44.808403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:44.808555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:44.826423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:44.827587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:44.841153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:44.899028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 563/1125] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:44.921473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b6dad-28bc-4a00-b79c-a329d066864f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:44.946162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:45.019171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b6dad-28bc-4a00-b79c-a329d066864f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:45.051351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:45.051983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:45.052263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:45.052535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] PATH_INFO: `/v3/roles/e0028b79d128480ba32f23616eb613c1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:45.053890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] RBAC: Authorizing `identity:get_role(role_id=e0028b79d128480ba32f23616eb613c1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:45.055249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:45.056572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 563/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:44 2022] GET /identity/v3/roles/e0028b79d128480ba32f23616eb613c1 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:45.062274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:45.161965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b1670-ab5d-43a5-8d7f-5552853d7b2e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:45.194813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:45.195424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:45.195701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:45.196121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] PATH_INFO: `/v3/users/8d919376a0624c4891db0ee5589d27e1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:45.197683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] RBAC: Authorizing `identity:get_user(user_id=8d919376a0624c4891db0ee5589d27e1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:45.199095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:45.200689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 564/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:45 2022] GET /identity/v3/users/8d919376a0624c4891db0ee5589d27e1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:45.206864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:45.293590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:45.302398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c2db-2576-4cd1-ae9f-af8f334ef8c2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:45.335634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:45.336245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:45.336582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:45.336849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:45.400395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:45.401812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:45.485929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:45.496120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 564/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:45.502335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:45.522493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:45.602780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd2005-de0a-4439-b2b0-ac3c5f82161c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:45.635514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:45.636140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:45.636417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:45.636691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] PATH_INFO: `/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:45.637783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] RBAC: Authorizing `identity:get_project(project_id=90c5e8d5a7b94ef1878f8b3e6cfef652)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:45.639192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c898a3-991b-438f-9085-f542a7c96318 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:45.640824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 565/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:45 2022] GET /identity/v3/projects/90c5e8d5a7b94ef1878f8b3e6cfef652 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:44:45.646968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:45.742706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd01f4-fb5d-4090-a2bd-15a13dd62781 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:45.774084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:45.774656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:45.774929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:45.775197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:45.775925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] RBAC: Adding query filter params (role.id=e0028b79d128480ba32f23616eb613c1, scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652, user.id=8d919376a0624c4891db0ee5589d27e1) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:45.776225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:45.777617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4875f-3237-462f-a461-8f22b61b17de None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:45.783107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 565/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:45 2022] GET /identity/v3/role_assignments?user.id=8d919376a0624c4891db0ee5589d27e1&scope.project.id=90c5e8d5a7b94ef1878f8b3e6cfef652&role.id=e0028b79d128480ba32f23616eb613c1 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:47.764553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a9fa-831f-4135-a2e8-5a3e1dd98692 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:47.765070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a9fa-831f-4135-a2e8-5a3e1dd98692 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:47.765163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a9fa-831f-4135-a2e8-5a3e1dd98692 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:47.766447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 566/1131] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:47.772049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:47.772437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:47.772596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:47.790052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:47.791221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:47.839187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:47.855027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 566/1132] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:47.877493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a9fa-831f-4135-a2e8-5a3e1dd98692 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:47.959192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:47.975156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a9fa-831f-4135-a2e8-5a3e1dd98692 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:48.008393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:48.009001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:48.009274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:48.009543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:48.010618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:48.012058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d95099-5733-40c2-a52f-3a988118233f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:48.013608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 567/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:48.019437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:48.115850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecd15e-8cf3-4daa-8b53-f8f0c0159346 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:48.147954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:48.148542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:48.148818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:48.149085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:48.149710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:48.151126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ed0b5-04d8-480d-8aed-0e66701c3650 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:48.247646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 567/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:44:48 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:50.249745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232161f7-092a-4fa4-a95b-9ab87787b700 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:50.250281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232161f7-092a-4fa4-a95b-9ab87787b700 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:50.250281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232161f7-092a-4fa4-a95b-9ab87787b700 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:50.251610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 568/1135] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:50.257504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:50.257950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:50.258042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:50.276125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:50.277290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:50.305152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:50.341530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 568/1136] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:50.365764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232161f7-092a-4fa4-a95b-9ab87787b700 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:50.461249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232161f7-092a-4fa4-a95b-9ab87787b700 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:50.493280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:50.493878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:50.494150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:50.494415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:50.495488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:50.496920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cefc1-fe5c-44de-be28-e8cb7aa92436 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:50.498427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 569/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:44:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:50.504087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:50.598761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef324e68-6926-490f-bd2b-efe29dda3bc7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:50.632042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:50.632632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:50.632909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:50.633179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:50.633817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:50.635231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81610aa-b7d7-419b-ad6b-7b810df58650 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:50.743450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 569/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:44:50 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:44:52.756296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334feddd-fce1-436e-a22e-c471311c4b5c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:52.756818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334feddd-fce1-436e-a22e-c471311c4b5c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:52.756818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334feddd-fce1-436e-a22e-c471311c4b5c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:52.758174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 570/1139] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:52.763188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:52.763626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:52.763725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:52.781229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:52.782389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:52.798946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:52.849095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 570/1140] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:52.872557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334feddd-fce1-436e-a22e-c471311c4b5c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:52.894076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:52.970030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334feddd-fce1-436e-a22e-c471311c4b5c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.002771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.003375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.003648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.003942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.013445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:53.017979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:53.019180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:53.020170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 571/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:53.025121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:53.120631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81d780-fed3-431b-9a75-9a3ee68c374a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.157943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.158451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.159270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.159544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.160236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:53.160539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:53.161900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:53.170435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 571/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:53.177009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:53.254371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:53.273149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581d865-390f-4a51-8174-84e8d4cd4900 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.305992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.306591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.306863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.307132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] PATH_INFO: `/v3/users/d325e9b66d57476da0894a98cb110b03` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.319440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=d325e9b66d57476da0894a98cb110b03)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:53.320880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:53.322429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 572/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:53 2022] GET /identity/v3/users/d325e9b66d57476da0894a98cb110b03 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:53.328374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:53.347695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:53.425118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efd75-2aa3-485c-9660-da0b99b60ec4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.456769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.457352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.457624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.457888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.522498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:53.523951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:53.540257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:53.618677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 572/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:53.624532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:53.720203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c51a18-5cc4-40a8-b27c-933fc398c1f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.753488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.754091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.754363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.754661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] PATH_INFO: `/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.755750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] RBAC: Authorizing `identity:get_project(project_id=31796a4eeced4a688fdcaad3dd0cf6ac)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:53.759707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c4a39-a9fe-402c-9663-8fea4f3b3660 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:53.759707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 573/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:53 2022] GET /identity/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:53.764855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:53.860676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407168-8efd-4960-83fd-bb11ee5bf661 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:53.876409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:53.894366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:53.894955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:53.895301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:53.895575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:53.896324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=31796a4eeced4a688fdcaad3dd0cf6ac, user.id=d325e9b66d57476da0894a98cb110b03) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:53.896630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:53.898039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be4f11-db85-4f47-8c71-6c8c4118836c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:53.903337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 573/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:53 2022] GET /identity/v3/role_assignments?user.id=d325e9b66d57476da0894a98cb110b03&scope.project.id=31796a4eeced4a688fdcaad3dd0cf6ac&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:55.853221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397ecc-01de-445a-8c89-ae49a0a03b20 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:55.853753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397ecc-01de-445a-8c89-ae49a0a03b20 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:55.853753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3397ecc-01de-445a-8c89-ae49a0a03b20 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:55.855014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 574/1147] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:55.860102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:55.860488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:55.860646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:55.879918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:55.881093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:55.944949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 574/1148] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:55.966660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3397ecc-01de-445a-8c89-ae49a0a03b20 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:55.976023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:56.064794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3397ecc-01de-445a-8c89-ae49a0a03b20 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:56.097114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:56.097716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:56.097990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:56.098255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:56.107838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:56.112898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:56.114021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:44:56.115105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 575/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:56.120247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:56.214775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1ba9-0d55-4cb7-ba11-93b89df5f48e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:56.225400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:56.247752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:56.248369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:56.248646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:56.248916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:56.249562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:56.249863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:56.251207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:56.258817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 575/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:56.264437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:56.327112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:56.360430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488c011-c720-425a-b56a-412a9c2ed1e2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:56.392612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:56.394022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:56.394462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:56.394794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] PATH_INFO: `/v3/users/d325e9b66d57476da0894a98cb110b03` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:56.395937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] RBAC: Authorizing `identity:get_user(user_id=d325e9b66d57476da0894a98cb110b03)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:56.397349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:56.398895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 576/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:56 2022] GET /identity/v3/users/d325e9b66d57476da0894a98cb110b03 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:56.404537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:56.500271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb84af-eabb-4b15-9c23-be253e12f728 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:56.532731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:56.533406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:56.534198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:56.534472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] PATH_INFO: `/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:56.535576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] RBAC: Authorizing `identity:get_project(project_id=31796a4eeced4a688fdcaad3dd0cf6ac)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:56.537036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe859ae-6617-4398-ba84-17d84cdbd408 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:56.538648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 576/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:56 2022] GET /identity/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:56.544338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:56.641222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6c00-6199-4356-bbd3-1212f2889264 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:56.674725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:56.675362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:56.675638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:56.675988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] PATH_INFO: `/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac/users/d325e9b66d57476da0894a98cb110b03/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:56.677810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] RBAC: Authorizing `identity:create_grant(project_id=31796a4eeced4a688fdcaad3dd0cf6ac, user_id=d325e9b66d57476da0894a98cb110b03, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:56.679288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89472c-b162-413c-8eb5-795322720238 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:56.681410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:56.759821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 577/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:44:56 2022] PUT /identity/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac/users/d325e9b66d57476da0894a98cb110b03/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:44:58.733071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42702cf-097b-441f-8d22-7b2420424cea None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:58.733581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42702cf-097b-441f-8d22-7b2420424cea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:58.733672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42702cf-097b-441f-8d22-7b2420424cea None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:58.734941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 577/1154] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:44:58.740492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:58.740882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:58.741042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:58.759080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:44:58.760282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:44:58.823448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 578/1155] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:44:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:44:58.846903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42702cf-097b-441f-8d22-7b2420424cea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:58.887986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:58.943813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42702cf-097b-441f-8d22-7b2420424cea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:58.975554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:58.976179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:58.976456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:58.976721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:58.986701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:44:58.991862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:44:58.992804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:44:58.993737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 578/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:44:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:44:58.998933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:59.092869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d6eb4-7df2-4171-a285-060808222679 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:59.115754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:59.125600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:59.127153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:59.127153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:59.127153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:59.127421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:59.127671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:59.129035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:59.136929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 579/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:44:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:59.144403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:59.235209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:59.238630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4ee4f-40d0-4d65-b3a9-3444e86c127d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:59.270371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:59.270970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:59.271244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:59.271512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] PATH_INFO: `/v3/users/d325e9b66d57476da0894a98cb110b03` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:59.272626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] RBAC: Authorizing `identity:get_user(user_id=d325e9b66d57476da0894a98cb110b03)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:59.274027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:59.275556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 579/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:44:59 2022] GET /identity/v3/users/d325e9b66d57476da0894a98cb110b03 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:59.281135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:59.378069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194fb7-9960-440f-9164-860c567b0981 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:59.410680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:59.411204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:59.411480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:59.411750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:59.462891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:59.475787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:59.477206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:59.569647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 580/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:44:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:44:59.575423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:59.673697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01209c-c9e7-4135-ad3e-d6bfc30e08a0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:59.705557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:59.706691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:59.706966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:59.707233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] PATH_INFO: `/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:59.708343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] RBAC: Authorizing `identity:get_project(project_id=31796a4eeced4a688fdcaad3dd0cf6ac)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:59.709753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13d43d-15bc-4b66-ad31-6cb908e2772c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:59.711356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 580/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:44:59 2022] GET /identity/v3/projects/31796a4eeced4a688fdcaad3dd0cf6ac => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:44:59.717706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:44:59.797008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:44:59.811136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b310f8-d023-4a9a-93bf-5fcb9f4c7b47 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:44:59.844197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:44:59.844772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:44:59.845043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:44:59.845305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:44:59.846013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=31796a4eeced4a688fdcaad3dd0cf6ac, user.id=d325e9b66d57476da0894a98cb110b03) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:44:59.846343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:44:59.847727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab45934-6514-46b7-85e1-88b915745600 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:44:59.853183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 581/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:44:59 2022] GET /identity/v3/role_assignments?user.id=d325e9b66d57476da0894a98cb110b03&scope.project.id=31796a4eeced4a688fdcaad3dd0cf6ac&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:01.877147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03f09-8f93-41a0-97b6-02e8066d688f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:01.877698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03f09-8f93-41a0-97b6-02e8066d688f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:01.877698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03f09-8f93-41a0-97b6-02e8066d688f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:01.879009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 581/1162] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:01.884304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:01.884699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:01.884851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:01.903056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:01.904247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:01.967738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 582/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:01.990246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf03f09-8f93-41a0-97b6-02e8066d688f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:02.025286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:02.088315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf03f09-8f93-41a0-97b6-02e8066d688f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:02.120808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:02.121394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:02.121672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:02.121944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:02.185653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:02.187086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:02.218683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:02.280240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 582/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:45:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:45:02.286717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:02.362119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:02.379545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109fa832-0c25-42d5-a64d-93d0c67ca991 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:02.411736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:02.412364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:02.412637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:02.412902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:02.419181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:02.420616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6852edfd-7932-4c84-b73f-fa7520418340 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:45:02.431834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:45:02.433063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:45:02.433948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 583/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:45:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:45:02.438964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:02.532536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf03d4e-3e5a-4667-a37b-6f81068f33a5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:02.565495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:02.566079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:02.566356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:02.566622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:02.567251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:02.567548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:02.568922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1435d9-e08e-47fc-ac12-54bb68a08bb6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:02.571102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:02.576122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 583/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Thu Aug 11 10:45:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:04.519641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c54c0-2c5e-4bbe-a61d-2351fed31069 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:04.520339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c54c0-2c5e-4bbe-a61d-2351fed31069 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:04.520339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c54c0-2c5e-4bbe-a61d-2351fed31069 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:04.521576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 584/1167] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:04.526567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6bf7-5fe0-4490-914a-632695fd7804 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:04.527010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6bf7-5fe0-4490-914a-632695fd7804 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:04.527119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a6bf7-5fe0-4490-914a-632695fd7804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:04.545384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-365a6bf7-5fe0-4490-914a-632695fd7804 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:04.546557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-365a6bf7-5fe0-4490-914a-632695fd7804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:04.611423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 584/1168] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:04.633145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c54c0-2c5e-4bbe-a61d-2351fed31069 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:04.731512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c54c0-2c5e-4bbe-a61d-2351fed31069 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:04.763195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda951-4c34-4866-af71-56ba03077add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:04.763800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda951-4c34-4866-af71-56ba03077add None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:04.764082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda951-4c34-4866-af71-56ba03077add None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:04.764351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda951-4c34-4866-af71-56ba03077add None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:04.764988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda951-4c34-4866-af71-56ba03077add None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:04.766322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda951-4c34-4866-af71-56ba03077add None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:04.863391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:04.869174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 585/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Thu Aug 11 10:45:04 2022] POST /identity/v3/domains => generated 267 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:45:06.880351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5678-791d-4867-a680-2be1b5f703ff None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:06.880856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5678-791d-4867-a680-2be1b5f703ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:06.880856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5678-791d-4867-a680-2be1b5f703ff None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:06.882181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 585/1170] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:06.887270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:06.887695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:06.887812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:06.905537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:06.906729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:06.969603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 586/1171] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:06.992075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e5678-791d-4867-a680-2be1b5f703ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:07.088693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e5678-791d-4867-a680-2be1b5f703ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:07.115907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:07.121360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:07.121968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:07.122243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:07.122512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] PATH_INFO: `/v3/domains/7c482255b67b4e1d96361644ebd40a09` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:07.132488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c482255b67b4e1d96361644ebd40a09)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:07.133863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec7813a-15f2-4c8b-8757-8722a7b0c755 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:07.135470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 586/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Aug 11 10:45:06 2022] GET /identity/v3/domains/7c482255b67b4e1d96361644ebd40a09 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:07.141122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:07.237426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf09839-19e6-459d-be5a-26f613566112 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:07.264281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:07.270227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:07.270822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:07.271097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:07.271364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:07.272015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:07.273418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f832a1-a5b8-4183-817d-c0249ac12cd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:07.353705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 587/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:45:07 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:45:09.353474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9e359-bd00-4099-a6e8-db6b5f970b4c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:09.354113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9e359-bd00-4099-a6e8-db6b5f970b4c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:09.354113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9e359-bd00-4099-a6e8-db6b5f970b4c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:09.355347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 587/1174] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:09.360450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:09.361402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:09.361524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:09.380340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:09.381508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:09.445706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 588/1175] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:09.469894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9e359-bd00-4099-a6e8-db6b5f970b4c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:09.567998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9e359-bd00-4099-a6e8-db6b5f970b4c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:09.600046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d210166-5682-47a5-b622-f042816c870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:09.600656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210166-5682-47a5-b622-f042816c870c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:09.600930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210166-5682-47a5-b622-f042816c870c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:09.601193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d210166-5682-47a5-b622-f042816c870c None admin] PATH_INFO: `/v3/domains/7c482255b67b4e1d96361644ebd40a09` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:09.602439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d210166-5682-47a5-b622-f042816c870c None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c482255b67b4e1d96361644ebd40a09)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:09.603734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d210166-5682-47a5-b622-f042816c870c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:09.605248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 588/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Thu Aug 11 10:45:09 2022] GET /identity/v3/domains/7c482255b67b4e1d96361644ebd40a09 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:09.611460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:09.705900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412622ad-ade6-41d2-9484-c82a5b62d0a2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:09.718551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:09.737973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:09.739892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:09.739892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:09.739892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:09.739892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:09.741221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe27321-4bd3-4ffc-b689-9af22f365ef9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:09.839204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 589/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:45:09 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:45:11.862059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6008-5532-418a-afb3-a03c34de2560 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:11.862578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6008-5532-418a-afb3-a03c34de2560 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:11.862578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e6008-5532-418a-afb3-a03c34de2560 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:11.863881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 589/1178] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:11.869009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:11.869458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:11.869561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:11.887499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:11.888706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:11.952077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 590/1179] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:11.974846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6008-5532-418a-afb3-a03c34de2560 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.032414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:12.074348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e6008-5532-418a-afb3-a03c34de2560 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:12.106615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:12.107248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:12.107524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:12.107815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:12.118586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:45:12.123159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:45:12.124353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:45:12.125623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 590/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:45:12.130324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.222110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:12.225699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05131b-2567-4408-b7eb-30b8872811e0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:12.258242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:12.258830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:12.259107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:12.259378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:12.260058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:12.260359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:12.261694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:12.269880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 591/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:45:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:12.275492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.371356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be38c6-ff76-4476-8414-17a61637530a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:12.387204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:12.403598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:12.404476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:12.404800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:12.405204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] PATH_INFO: `/v3/users/f19058068070480db88c30ac0f8f77c8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:12.418520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] RBAC: Authorizing `identity:get_user(user_id=f19058068070480db88c30ac0f8f77c8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:12.419974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:12.421538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 591/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:45:12 2022] GET /identity/v3/users/f19058068070480db88c30ac0f8f77c8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:12.427119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.522234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41bdd2-5a5d-42c5-80a5-a6db36a69ab3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:12.553367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec78690-c167-4699-acd2-72357754080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:12.553950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78690-c167-4699-acd2-72357754080f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:12.554225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78690-c167-4699-acd2-72357754080f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:12.554494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78690-c167-4699-acd2-72357754080f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:12.581126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:12.664169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec78690-c167-4699-acd2-72357754080f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:12.665543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec78690-c167-4699-acd2-72357754080f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:12.759145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 592/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:45:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:45:12.764931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.860521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c537-05f6-417c-b495-47b97ab7a332 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:12.899890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:12.900508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:12.900773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:12.901048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] PATH_INFO: `/v3/projects/10ee2470c64c4608ac559947985d5952` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:12.902143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] RBAC: Authorizing `identity:get_project(project_id=10ee2470c64c4608ac559947985d5952)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:12.903562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96ff79-cf82-424e-9354-a1b2a31a3df7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:12.905213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 592/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:45:12 2022] GET /identity/v3/projects/10ee2470c64c4608ac559947985d5952 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:12.911802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec78690-c167-4699-acd2-72357754080f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:12.967417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:13.005549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec78690-c167-4699-acd2-72357754080f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:13.038526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:13.039110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:13.039386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:13.039654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:13.040396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=10ee2470c64c4608ac559947985d5952, user.id=f19058068070480db88c30ac0f8f77c8) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:13.040700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:13.042097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b4484-12cb-48ac-9cba-5c4dea516058 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:13.047219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 593/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:45:12 2022] GET /identity/v3/role_assignments?user.id=f19058068070480db88c30ac0f8f77c8&scope.project.id=10ee2470c64c4608ac559947985d5952&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:14.993644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a2e4-5826-4690-af1d-8966e5dc7f58 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:14.994181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a2e4-5826-4690-af1d-8966e5dc7f58 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:14.994181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a2e4-5826-4690-af1d-8966e5dc7f58 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:14.995544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 593/1186] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:15.000555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.001505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.001646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.019581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:15.020785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:15.084753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 594/1187] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:15.106419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a2e4-5826-4690-af1d-8966e5dc7f58 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:15.151617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:15.203051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a2e4-5826-4690-af1d-8966e5dc7f58 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:15.236041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:15.236646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.236920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.237186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.246870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:45:15.251582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:45:15.252699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:45:15.253721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 594/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:45:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:45:15.258435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:15.310817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:15.353701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42150061-c5c6-4d95-8f02-0439249f697e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:15.385282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:15.385859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.386135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.386443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.387086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:15.387382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:15.388744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:15.396874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 595/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:45:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:15.402533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:15.500163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d48b7-ff9c-4549-bd91-ae90f7482c40 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:15.502824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:15.532932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:15.533536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.533807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.534068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] PATH_INFO: `/v3/users/f19058068070480db88c30ac0f8f77c8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.535155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] RBAC: Authorizing `identity:get_user(user_id=f19058068070480db88c30ac0f8f77c8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:15.536696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:15.538228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 595/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:45:15 2022] GET /identity/v3/users/f19058068070480db88c30ac0f8f77c8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:15.543664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:15.640479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a11082-cae9-4fb6-81e7-360814756390 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:15.655526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:15.672788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:15.673889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.674163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.674432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] PATH_INFO: `/v3/projects/10ee2470c64c4608ac559947985d5952` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.675511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] RBAC: Authorizing `identity:get_project(project_id=10ee2470c64c4608ac559947985d5952)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:15.676946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8c6f4-4601-49fb-b192-ef6cfbf6a05b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:15.678537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 596/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:45:15 2022] GET /identity/v3/projects/10ee2470c64c4608ac559947985d5952 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:15.684465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:15.779083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c134c1d-920e-4f26-8336-eacef9ab6264 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:15.811958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:15.812592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:15.812869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:15.813136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] PATH_INFO: `/v3/projects/10ee2470c64c4608ac559947985d5952/users/f19058068070480db88c30ac0f8f77c8/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:15.815023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] RBAC: Authorizing `identity:create_grant(project_id=10ee2470c64c4608ac559947985d5952, user_id=f19058068070480db88c30ac0f8f77c8, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:15.816471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e96d0-54c3-4c0e-9442-1d5c4245c380 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:15.878874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 596/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:45:15 2022] PUT /identity/v3/projects/10ee2470c64c4608ac559947985d5952/users/f19058068070480db88c30ac0f8f77c8/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:45:17.844973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0bcb8-9b5d-4ff4-87cc-390012f3eed4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:17.845527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0bcb8-9b5d-4ff4-87cc-390012f3eed4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:17.845527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0bcb8-9b5d-4ff4-87cc-390012f3eed4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:17.846851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 597/1193] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:45:17.851997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:17.852384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:17.852496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:17.873171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:45:17.874368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:45:17.902743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:17.949570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 597/1194] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:45:17.974854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0bcb8-9b5d-4ff4-87cc-390012f3eed4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.081518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0bcb8-9b5d-4ff4-87cc-390012f3eed4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:18.118664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:18.119314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:18.119588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:18.119888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:18.131919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:45:18.136951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:45:18.137974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:45:18.139213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 598/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:45:18.144745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.242508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470d2b8-146c-4e0d-aba3-d09695b90e47 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:18.276419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:18.277015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:18.277290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:18.277559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:18.278216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:18.278520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:18.279907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:18.288381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 598/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:18.294210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.342061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:18.398380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ac2d0-745e-4244-a674-4734aad5b71a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:18.434005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:18.434626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:18.434907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:18.435182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] PATH_INFO: `/v3/users/f19058068070480db88c30ac0f8f77c8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:18.436323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] RBAC: Authorizing `identity:get_user(user_id=f19058068070480db88c30ac0f8f77c8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:18.437749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:18.439853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 599/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Thu Aug 11 10:45:18 2022] GET /identity/v3/users/f19058068070480db88c30ac0f8f77c8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:18.445691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.471608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:18.546353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c10ea0-7b80-491c-816d-1b4cd252997e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:18.579297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:18.579964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:18.580247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:18.580511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:18.649654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:18.651029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:18.675436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:18.746330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 599/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:45:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:45:18.752288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.846911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a01b1-ccd2-4df0-bbab-505738f88a09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:18.858756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:18.878029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:18.878630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:18.878903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:18.879189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] PATH_INFO: `/v3/projects/10ee2470c64c4608ac559947985d5952` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:18.880291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] RBAC: Authorizing `identity:get_project(project_id=10ee2470c64c4608ac559947985d5952)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:18.881733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2a677-91aa-4b49-af42-e8f8d51fcfdc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:18.883308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 600/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Thu Aug 11 10:45:18 2022] GET /identity/v3/projects/10ee2470c64c4608ac559947985d5952 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:18.889371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:45:18.985077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f880d29-4c76-44de-a570-9bd204b94ecf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:45:19.007076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:19.017702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:45:19.018300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:45:19.018637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:45:19.018910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:45:19.019622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=10ee2470c64c4608ac559947985d5952, user.id=f19058068070480db88c30ac0f8f77c8) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:45:19.019960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:45:19.021373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08f0f-bf1b-4461-95d4-7fc5af680322 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:45:19.026886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 600/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:45:18 2022] GET /identity/v3/role_assignments?user.id=f19058068070480db88c30ac0f8f77c8&scope.project.id=10ee2470c64c4608ac559947985d5952&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:45:28.906792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:45:29.033366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:36.777332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcb7ed-56d4-4704-a0f8-0c225cb4967c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:36.777920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcb7ed-56d4-4704-a0f8-0c225cb4967c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:36.777920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcb7ed-56d4-4704-a0f8-0c225cb4967c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:36.779256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 601/1201] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:36.784617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:36.785065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:36.785190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:36.803203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:36.804404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:36.869596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 601/1202] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:36.894510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcb7ed-56d4-4704-a0f8-0c225cb4967c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:36.994792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dcb7ed-56d4-4704-a0f8-0c225cb4967c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:37.028277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:37.028886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:37.029166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:37.029439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:37.039014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:37.040463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:37.054452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:37.055602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:37.059204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:37.059204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:37.059204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:37.059204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 602/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:37.062059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:37.159665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047062-ecd3-47ea-a889-30b616d95c5f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:37.187853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:37.193324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:37.193916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:37.194190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:37.194457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:37.195083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:37.195383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:37.196756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69cba-e53f-4d9a-958d-e9d838afd871 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:37.208349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 602/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:37.214017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:37.265209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:37.310460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b401a2-a86e-4325-b88f-6e3d5475e335 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:37.342438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:37.343559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:37.343864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:37.344137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:37.344769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:37.346167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdb762-dad0-460e-8eef-7a97aa645c52 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:37.476916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 603/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Thu Aug 11 10:46:37 2022] POST /identity/v3/users => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:46:39.472455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b771-ee2c-4801-9972-fb06a3312af8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:39.473019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b771-ee2c-4801-9972-fb06a3312af8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:39.473019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3b771-ee2c-4801-9972-fb06a3312af8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:39.474325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 603/1206] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:39.479384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:39.481200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:39.481200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:39.497731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:39.499406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:39.562924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 604/1207] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:39.587575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3b771-ee2c-4801-9972-fb06a3312af8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:39.685793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3b771-ee2c-4801-9972-fb06a3312af8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:39.718249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:39.718867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:39.719139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:39.719403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:39.729612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:39.734807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:39.735946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:39.737252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 604/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:46:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:39.741920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:39.780757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:39.842444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0d2d4-3c56-421d-a253-8176926f0acc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:39.875810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:39.876411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:39.876690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:39.876990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:39.877649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:39.877949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:39.879297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:39.886874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 605/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:39.892622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:39.942171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:39.991085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e9f11-17b5-41cd-9863-ae788cc96cf5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.022703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.023543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.023750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.023889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.034287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.035715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:40.049741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:40.055443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:40.056492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:40.056492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:40.056492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:40.056492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 605/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:40.056750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:40.159975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:40.162936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986b7f2-980e-4c69-840d-3823aaecde37 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.195345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.195971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.196253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.196525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.197157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:40.197456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.198804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:40.210323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 606/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:40.216287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:40.313949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca1415-4b41-43d3-a6e5-23c2e22cd740 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.315399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:40.346785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.347400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.347680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.347977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.355022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.356480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:46:40.367128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:40.368328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:40.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 606/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:40.374522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:40.484697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd34df-2009-41c0-863c-69deff1eaff5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.518473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.519081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.519355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.519629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.520377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:40.520760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.522176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:40.538941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 607/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:46:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:40.544691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:40.644071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021446-8707-456e-99ec-66a6a3eabd0a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.676861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.677461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.677739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.678008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.707125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:40.743249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.744788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:40.847000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 607/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:46:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:46:40.853036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:40.872433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:40.951076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb919-a70e-42af-b8c5-354336edae38 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:40.984081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:40.984711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:40.985194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:40.985629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:40.995999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:40.997423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:41.011340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:41.013785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:41.015023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:41.015023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:41.015023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:41.015023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 608/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:41.018224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:41.078302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:41.116032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac030507-c58a-4f64-bb75-3a7a2ea6e812 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:41.147600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:41.148213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:41.148488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:41.148755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:41.149373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:41.149669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:41.151022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:41.162202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 608/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:41.167836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:41.234334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:41.263768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907a2ee-d8b3-43a4-90fd-69af555cf511 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:41.296990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:41.300425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:41.300425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:41.300425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:41.304670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:41.306094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:46:41.316889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:41.317963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:41.319105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 609/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:41.323790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:41.422079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985a468-2374-45ef-b4ba-67e6bf2000e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:41.443711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:41.455253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:41.455892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:41.456172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:41.456443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:41.457378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:41.457681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:41.459100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd85a7-8950-4ea7-8140-988968053ea3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:41.471148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 609/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:41.477257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:41.574396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c515341-9ed8-4a9c-9b93-b677925c972b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:41.602595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:41.608366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:41.608992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:41.609270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:41.609544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:41.610262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=25af3fcbdf174cc2bf22fb99c83575f6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:41.610570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:41.612019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4f28d-3a9b-427e-8c6e-88d593f1f558 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:41.617357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 610/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:46:41 2022] GET /identity/v3/role_assignments?user.id=25af3fcbdf174cc2bf22fb99c83575f6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:43.576029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e7b47-1359-4b1e-88d5-45c9cded7676 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:43.576587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e7b47-1359-4b1e-88d5-45c9cded7676 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:43.576587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e7b47-1359-4b1e-88d5-45c9cded7676 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:43.577932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 610/1220] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:43.583254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:43.586619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:43.586619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:43.602854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:43.604090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:43.669706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 611/1221] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:46:43.692701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e7b47-1359-4b1e-88d5-45c9cded7676 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:43.792444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e7b47-1359-4b1e-88d5-45c9cded7676 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:43.825426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:43.826079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:43.826300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:43.826562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:43.836582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:43.841272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:43.842970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:43.844108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 611/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:43.848716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:43.944120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c1dab-12b7-401d-a413-0879d54f55d2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:43.966766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:43.977641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:43.978239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:43.978525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:43.978796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:43.979449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:43.979753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:43.981230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:43.989361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 612/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:43.995480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.038217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:44.092404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed91a1-91a7-45bc-9982-bd81015052f6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.124940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.125559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.125833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.126101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.136036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.137469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:44.152936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:44.154069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:44.155708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:44.155708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:44.155708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:44.155708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 612/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:44.160782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.258658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7655-d386-4515-bd7e-d4f1c5feb672 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.290477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.291070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.291352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.291624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.292284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:44.292588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.293931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.305620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 613/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:44.311907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.411388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:44.417902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48304174-e695-4fbb-b078-cdb59bef7732 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.457838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.458455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.458736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.459010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.466234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.468308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:46:44.478910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:44.489711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:44.490841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 613/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:44.491500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.531645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:44.582753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a222-35f1-4f37-960a-a64a230c907d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.616324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.616928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.617204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.617471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.618184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:44.618489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.619935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.636803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 614/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:44.643107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.742391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62453844-16ed-4e13-93f2-e53761cd109f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.774326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.774940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.775222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.775492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.786447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.787934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:44.802390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:44.803549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:44.804903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:44.804903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:44.804903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:44.804903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 614/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:44.809225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.905026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ce21f-2ab3-474a-a59b-9c7d9f1ae4bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:44.907248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:44.937673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:44.938304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:44.938578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:44.938844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:44.939469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:44.939791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:44.941125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:44.951968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 615/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:44.957818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:44.967112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:45.056070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a119c-3303-4a58-bac6-ae2555d8afe3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:45.088685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:45.089303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:45.089579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:45.089848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:45.096374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:45.097804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:46:45.109550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:45.110602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:45.111717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 615/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:45.116530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:45.219246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d4b9f-b3bc-4383-a99d-c79de1460554 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:45.252067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333a500-edff-4403-be43-f399243f6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:45.252738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333a500-edff-4403-be43-f399243f6206 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:45.253019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333a500-edff-4403-be43-f399243f6206 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:45.253289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333a500-edff-4403-be43-f399243f6206 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:45.253967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333a500-edff-4403-be43-f399243f6206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:45.254268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333a500-edff-4403-be43-f399243f6206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:45.255810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333a500-edff-4403-be43-f399243f6206 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:45.267414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 616/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:45.273656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:45.338060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:45.371409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5f773-79d1-4cd5-bf38-1a80204f807a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:45.403845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:45.404831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:45.405108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:45.405378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/25af3fcbdf174cc2bf22fb99c83575f6/roles/b92231fbb0ea429db746a65bc582ba9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:45.419603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=25af3fcbdf174cc2bf22fb99c83575f6, role_id=b92231fbb0ea429db746a65bc582ba9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:45.421059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a2023-4c7d-4f5d-b780-29a636c81947 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:45.493867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 616/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:46:45 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/25af3fcbdf174cc2bf22fb99c83575f6/roles/b92231fbb0ea429db746a65bc582ba9a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:46:47.473222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ee0a-635c-4e4d-8b3b-8a1ffa6e28f8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:47.473777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ee0a-635c-4e4d-8b3b-8a1ffa6e28f8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:47.473777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2ee0a-635c-4e4d-8b3b-8a1ffa6e28f8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:47.475140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 617/1233] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:47.480533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:47.480933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:47.481084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:47.499111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:47.500325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:47.565519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 617/1234] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:47.589697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2ee0a-635c-4e4d-8b3b-8a1ffa6e28f8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:47.614883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:47.689974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2ee0a-635c-4e4d-8b3b-8a1ffa6e28f8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:47.722727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:47.723392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:47.723625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:47.723929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:47.733800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:47.738519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:47.743941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:47.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 618/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Thu Aug 11 10:46:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:47.745907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:47.791544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:47.843181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e9f3c-f2ba-45f0-ad0a-16b41bf29f23 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:47.877182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:47.877775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:47.878053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:47.878325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:47.878981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:47.879284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:47.880683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:47.888322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 618/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Thu Aug 11 10:46:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:47.894671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:47.978316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:47.993666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b47f81-bdc4-4911-86f7-ef784087f098 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.026405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.027019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.027297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.027569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.037264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.038755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:48.054236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:48.055404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:48.056577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:48.056577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:48.056577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:48.056577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 619/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:48.061805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:48.162916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6fdca-d247-46d4-ba4d-00d53f46dd77 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.173674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:48.197560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.198155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.198429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.198698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.199324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:48.199622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.201021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:48.212440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 619/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:48.218132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:48.318056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97edae-28f5-4175-822c-63ce3f0b5b86 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.350646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.351252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.351531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.351828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.354376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:48.359633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.361226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:46:48.371797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:48.373759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:48.374782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 620/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:48.378774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:48.483676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab398ba-469d-4fa8-a957-e47330afa1ed None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.517563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.518156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.518433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.518703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.519415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:48.519715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.521202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:48.537549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 620/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:46:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:48.543389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:48.642595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0815e-f139-4352-83eb-112330e81175 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.674959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.675575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.675886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.676157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.735654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:48.741907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.743322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:48.837694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 621/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:46:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:46:48.844228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:48.870827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:48.940863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a026-6631-42c7-b0a3-82cf4ce1e898 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:48.974331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:48.974954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:48.975251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:48.975562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:48.985438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:48.987038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:49.001678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:49.002801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:49.003973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:49.003973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:49.003973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:49.003973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 621/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 10:46:49.008681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:49.100515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:49.106095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1be8a-f020-412e-b252-c710e2b575dc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:49.140802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:49.141319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:49.141595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:49.141861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:49.142492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:49.142811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:49.144175 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:49.154880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 622/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:49.160748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:49.237125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:49.258099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95513cdc-8712-42a1-99d9-c5b2de973767 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:49.290175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:49.290784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:49.291058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:49.291323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:49.297724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:49.299137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:46:49.311380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:49.316890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:49.318211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 622/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:49.318812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:49.416042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5e20c-1078-4b16-ac7c-fc678c621a48 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:49.451898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:49.452495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:49.454032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:49.454032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:49.454032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:49.454032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:49.455414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeddafc-4fae-4690-a809-93a09cc07f60 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:49.467470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 623/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:49.473513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:49.571534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a0a07-50d6-4c57-876d-c30a42626585 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:49.605684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:49.606323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:49.606599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:49.606891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:49.607599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] RBAC: Adding query filter params (role.id=b92231fbb0ea429db746a65bc582ba9a, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=25af3fcbdf174cc2bf22fb99c83575f6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:49.607933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:49.609349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4e12-44e3-4352-90af-d4515624f18f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:49.615064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 623/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:46:49 2022] GET /identity/v3/role_assignments?user.id=25af3fcbdf174cc2bf22fb99c83575f6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=b92231fbb0ea429db746a65bc582ba9a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:51.679788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f5fd-a7b7-4fc5-9b67-2318d7272e0c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:51.680329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f5fd-a7b7-4fc5-9b67-2318d7272e0c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:51.680329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f5fd-a7b7-4fc5-9b67-2318d7272e0c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:51.681702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 624/1247] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:51.687437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:51.687908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:51.688043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:51.694232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:51.706219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:51.710647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:51.771883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 624/1248] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:51.795440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861f5fd-a7b7-4fc5-9b67-2318d7272e0c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:51.813775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:51.894183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861f5fd-a7b7-4fc5-9b67-2318d7272e0c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:51.927191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:51.927835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:51.928117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:51.928388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:51.937988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:51.942609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:51.943807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:51.944885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 625/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:51.949756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.046306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37296cd-1805-416e-bcd5-b15bd8563ee6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.074434 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:52.080734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.080734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.080734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.081302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.081762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:52.082058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.083394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:52.091133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 625/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:46:52.096845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.171699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:52.194093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7ca53-7dde-42a3-a324-fe2d7bf26940 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.227233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.227867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.228145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.228413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.238474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.240085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:52.253963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:52.255130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:52.256329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:52.256329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:52.256329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:52.261298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 626/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:52.261432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.357488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d5e83-5c1f-4a9c-9250-5300601602bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.390105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.390717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.390989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.391256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.391922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:52.392275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.393551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:52.404882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 626/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:52.410657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.509745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b726-4c81-4c24-9b00-954b3c0621d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.534348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:52.541795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.542404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.542679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.542949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.550541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.552021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:46:52.563122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:52.564306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:52.565460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 627/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:52.570007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.626299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:52.668184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e121373-d6f6-4ae0-b008-08f11b222c6a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.701668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.702259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.702512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.702778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.703482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:52.703804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.705213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:52.722705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 627/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:46:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:52.728584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:52.825121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27d9a-1c5e-4ac7-af44-9006a2faad08 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:52.857635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:52.858214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:52.858488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:52.858752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:52.912993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:52.924583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:52.926029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.019687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 628/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:46:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:46:53.025511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:53.122877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fe138-0a18-4c22-8f12-a0f3b2a5c7bc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:53.135149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:53.155714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:53.158626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:53.158626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:53.158626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:53.166321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:53.167726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:53.181317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:53.182489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:53.184587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:53.184587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:53.184587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:53.184587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 628/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:53.188422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:53.273020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:53.284878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b686c1-2f8b-45cc-bc94-ac19d0b01d71 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:53.316561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:53.317149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:53.317425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:53.317696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:53.318320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:53.318616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:53.319979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.331322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 629/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:53.336926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:53.434000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff95628-03a0-4a8a-8c5b-2cf0d849a0a1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:53.465940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:53.466542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:53.466813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:53.467081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:53.473409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:53.474810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:46:53.485945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:53.487015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:53.488397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 629/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:53.492901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:53.591314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d094c9-7802-4a29-a665-6749f8466264 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:53.624333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:53.624931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:53.625206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:53.625474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:53.630682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:53.630682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:53.630682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b8755-d25a-4e2f-a6ab-bea4723bea2e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.639376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 630/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:53.645724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:53.672582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:53.744140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995cf6-88b8-4afb-a253-f444c0230054 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:53.777388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:53.777980 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:53.778258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:53.778531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:53.779338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=25af3fcbdf174cc2bf22fb99c83575f6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:53.779554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:53.781009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ea430-2ceb-4b9a-a2e7-781d9a381773 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:53.787084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 630/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:46:53 2022] GET /identity/v3/role_assignments?user.id=25af3fcbdf174cc2bf22fb99c83575f6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:55.727682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464a7a-5db4-48bb-9ff7-d808e7b5af51 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:55.728259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464a7a-5db4-48bb-9ff7-d808e7b5af51 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:55.728259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464a7a-5db4-48bb-9ff7-d808e7b5af51 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:55.729582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 631/1261] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:55.735262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:55.735712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:55.735834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:55.754027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:55.755209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:55.825798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 631/1262] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:55.848664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464a7a-5db4-48bb-9ff7-d808e7b5af51 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:55.859992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:55.944211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464a7a-5db4-48bb-9ff7-d808e7b5af51 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:55.976113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:55.976717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:55.976995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:55.977261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:55.987343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:55.991925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:55.993009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:55.994301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 632/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:55.998959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.016923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:56.096198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2263a6a-39f8-47fd-849b-0354b09aa6b8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.137106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.138111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.138389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.138659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.139320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:56.139621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.141088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.149053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 632/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:56.155685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.220165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:56.251326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc251e50-da47-4970-be09-510fee67b825 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.284082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.284694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.284969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.285237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.296149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.297571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:56.310764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:56.311898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:56.313241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:56.313241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:56.313241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:56.313241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 633/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:56.317643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.396696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:56.415191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c395b6-94e7-4cda-a1eb-cfbb29b7304b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.449495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.450132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.450405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.450672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.451295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:56.451594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.452967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.464352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 633/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:56.470209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.574449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733a5cc-d914-4dfb-a8f6-f326fb110394 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.594242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:56.608514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.609133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.609408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.609674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.616847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.618244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:46:56.628736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:56.629854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:56.631046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 634/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:56.635922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.733612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9608a-c71b-41ac-9c89-417ab19eca2d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.767723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.768381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.768607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.768873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.769585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:56.769883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.771336 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.779724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:56.788196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 634/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:56.794134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:56.891756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc64e-6a68-4caf-88b7-9e6ee53628ec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:56.924210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:56.924818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:56.925096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:56.925366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:56.935475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:56.936929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:46:56.950772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:56.952403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:46:56.953571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:46:56.953571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:46:56.953571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:56.953571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 635/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:56.958025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:57.055109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb1f7d-c843-490e-8582-a7701e430b37 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:57.088232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:57.088820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:57.089325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:57.089325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:57.089955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:57.090249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:57.092202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:57.102918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 635/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:57.108593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:57.139608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:57.205414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37019c2c-053a-4fa7-a151-6d984e90d06e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:57.239095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:57.239747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:57.240057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:57.240331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:57.247249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:57.248683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:46:57.259674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:57.260789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:46:57.261977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 636/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:57.267361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:57.325432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:57.365129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a5afb-69bd-4bcb-8925-99c72aafcb9d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:57.398827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:57.399424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:57.399700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:57.400005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:57.400685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:46:57.400986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:57.402399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d269b-3ea5-44e6-8ad2-5a3a4592851e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:57.414769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 636/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:46:57.421382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:57.510334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:57.518553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df149c1c-3a4e-4061-9c69-674305e294bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:57.552219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:57.552854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:57.553131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:57.553403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] PATH_INFO: `/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/25af3fcbdf174cc2bf22fb99c83575f6/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:57.555250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] RBAC: Authorizing `identity:create_grant(project_id=17ebc11defe74b93b10eaac92f9dacd5, user_id=25af3fcbdf174cc2bf22fb99c83575f6, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:46:57.556682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee1118-6edb-44c6-9a05-019fa2f23318 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:46:57.663946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 637/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Thu Aug 11 10:46:57 2022] PUT /identity/v3/projects/17ebc11defe74b93b10eaac92f9dacd5/users/25af3fcbdf174cc2bf22fb99c83575f6/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:46:59.624206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d924-0b7f-4f9e-b239-af4eb3b045d7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:59.624708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d924-0b7f-4f9e-b239-af4eb3b045d7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:59.624819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801d924-0b7f-4f9e-b239-af4eb3b045d7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:59.626116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 637/1274] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:46:59.631055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:59.631489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:59.631584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:59.649777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:46:59.650978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:46:59.715308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 638/1275] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:46:59.738146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801d924-0b7f-4f9e-b239-af4eb3b045d7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:59.840430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801d924-0b7f-4f9e-b239-af4eb3b045d7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:46:59.857138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:59.873747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:46:59.874361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:46:59.874639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:46:59.874911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:46:59.884800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 10:46:59.890071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:46:59.890987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:46:59.892071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 638/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Thu Aug 11 10:46:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:46:59.897153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:46:59.975990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:46:59.993925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2bfb0-300c-41b0-b26e-05dcae8851f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.026517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.027107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.027387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.027662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.028351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:00.028660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.030010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.037845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 639/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Thu Aug 11 10:46:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:00.044062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:00.142503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddacdba-59cb-4359-98a3-4e975aba2151 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.174745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.175358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.175639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.175942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.186571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.188040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:47:00.201682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:00.205743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:47:00.206673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:47:00.206673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:47:00.206673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:00.206673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 639/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:00.208762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:00.306570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80678d9b-3d61-49f0-b2b2-5cd43140b14c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.338906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.339497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.339798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.340583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.341215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:00.341517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.342863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.354089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 640/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:00.360302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:00.391399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:00.458104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766fe5-8604-4b9b-b8c4-347c8c60317d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.492750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.493571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.493853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.494149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.502428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.504053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 10:47:00.515703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:00.521473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:00.523938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 640/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:00.524783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:00.526445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:00.619619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb22421-8c94-4b16-bb62-30242a9ffbfc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.653005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.653591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.653863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.654130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.654833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:00.655129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.656624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.673422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 641/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:47:00.679293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:00.768197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:00.778060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea32-59e7-41ab-8247-cddace93e08b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:00.809536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:00.810126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:00.810421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:00.810730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:00.883700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:00.885168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:00.970630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:00.980397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 641/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 10:47:00.986238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:01.047137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:01.084402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343346bc-692d-4d53-9450-f97b9281c34a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:01.116829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:01.117435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:01.117706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:01.117970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:01.127965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:01.129404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 10:47:01.143663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:01.151421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 10:47:01.153188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 10:47:01.153188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 10:47:01.153188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:47:01.153188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 642/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:01.153488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:01.246809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bb735-b763-41a1-9ee8-a43e13e41c08 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:01.279694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:01.280311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:01.280589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:01.280861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:01.281486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:01.281785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:01.283139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:01.294627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 642/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Thu Aug 11 10:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:01.308611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:01.396923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211552a-7931-4715-b818-8548c61bd3f0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:01.409988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:01.429225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:01.429833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:01.430110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:01.430378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:01.437288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:01.438704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 10:47:01.449370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:01.450292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 11 10:47:01.451325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 643/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Thu Aug 11 10:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:01.456486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:01.504560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:01.550952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badcedf-d352-49f6-985b-3bed112b4597 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:01.583265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:01.583803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:01.584083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:01.584383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:01.585069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:01.585368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:01.586780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bce714-9a1c-487b-87a2-787e6ac4d6d0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:01.599254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 643/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Thu Aug 11 10:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:01.605319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:01.703506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8b2f7-c01c-4261-b6d6-ae288611378b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:01.736473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:01.737105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:01.737382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:01.737656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:01.738375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] RBAC: Adding query filter params (role.id=9ff91fa864bb40e18f65458ec16b6f07, scope.project.id=17ebc11defe74b93b10eaac92f9dacd5, user.id=25af3fcbdf174cc2bf22fb99c83575f6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:01.738685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:01.740127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f03f-faa3-4a92-9e38-a4c48ba818ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:01.745872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 644/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Thu Aug 11 10:47:01 2022] GET /identity/v3/role_assignments?user.id=25af3fcbdf174cc2bf22fb99c83575f6&scope.project.id=17ebc11defe74b93b10eaac92f9dacd5&role.id=9ff91fa864bb40e18f65458ec16b6f07 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:03.736048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08169-d012-4b3d-a131-3f7731a5f753 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:03.736599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08169-d012-4b3d-a131-3f7731a5f753 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:03.736693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d08169-d012-4b3d-a131-3f7731a5f753 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:03.738030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 644/1288] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:47:03.743366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:03.743816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:03.743958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:03.762176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:47:03.763394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:47:03.792479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:03.830898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 645/1289] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:47:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:47:03.853966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d08169-d012-4b3d-a131-3f7731a5f753 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:03.953900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d08169-d012-4b3d-a131-3f7731a5f753 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:03.964553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:03.987423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:03.988063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:03.988335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:03.988598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:03.989167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:03.990509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:03.999513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:04.000820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:04.002009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 645/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:47:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:04.006697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:04.105577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103e81-1635-40b5-97b4-9f8c39f094c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:04.139983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:04.140590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:04.140864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:04.141133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:04.141788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:04.142089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:04.143465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bad64a-59e6-4722-a3dc-472aca2c614d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:04.148476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 646/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Aug 11 10:47:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:04.154865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:04.260296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05049-2af5-4a08-b09e-83cd3df4be72 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:04.293548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:04.294151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:04.294427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:04.294704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:04.295321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:04.295623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:04.297004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1eb72f-d43d-4d89-a9a3-423b8da9e6b4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:04.301451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 646/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Aug 11 10:47:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:06.301242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a5b83-df9c-4232-a8a1-8033af15b2a8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:06.301779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a5b83-df9c-4232-a8a1-8033af15b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:06.301779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a5b83-df9c-4232-a8a1-8033af15b2a8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:06.303129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 647/1293] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:47:06.309108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ad760-d913-4057-8b59-d5b2bebea323 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:06.309937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ad760-d913-4057-8b59-d5b2bebea323 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:06.309937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ad760-d913-4057-8b59-d5b2bebea323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:06.330314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-080ad760-d913-4057-8b59-d5b2bebea323 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:47:06.331472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-080ad760-d913-4057-8b59-d5b2bebea323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:47:06.359855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:06.396639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 647/1294] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:47:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 10:47:06.419187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a5b83-df9c-4232-a8a1-8033af15b2a8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:06.451466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:06.524530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a5b83-df9c-4232-a8a1-8033af15b2a8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:06.557344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:06.557872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:06.558660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:06.558923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:06.559538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:06.560918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e48696-e9a6-4aa2-8003-35c8f030d2f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:06.646935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 648/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Thu Aug 11 10:47:06 2022] POST /identity/v3/services => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:47:08.634933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbd53-e06e-4e96-b941-3b70f213db35 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:08.635432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbd53-e06e-4e96-b941-3b70f213db35 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:08.635564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbd53-e06e-4e96-b941-3b70f213db35 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:08.636973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 648/1296] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:47:08.642300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:08.642713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:08.642874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:08.660948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:47:08.662192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:47:08.733152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 649/1297] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:47:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:47:08.755192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbbd53-e06e-4e96-b941-3b70f213db35 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:08.859086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbbd53-e06e-4e96-b941-3b70f213db35 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:08.891992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:08.892609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:08.892886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:08.893158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:08.893738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:08.895093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:08.904243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:08.905265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:08.906239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 649/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:47:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:08.911427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:08.929042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:09.005324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553de-f4c4-4e36-8edd-94e064fb0696 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:09.038465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:09.039058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:09.039335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:09.039606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:09.040255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:09.040555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:09.041898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:09.047274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 650/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Aug 11 10:47:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:09.052866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:09.084261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:09.156563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d84d61-7e17-4cee-8b4f-da27c4504ab6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:09.188887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:09.189482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:09.189763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:09.190033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:09.190723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] RBAC: Adding query filter params (interface=public, service_id=b6e9d59f4fa541028c59446dc7715f7c, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:09.191028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:09.192418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454dd44-c61e-4491-a070-7bc152d1238e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:09.197301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 650/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Thu Aug 11 10:47:09 2022] GET /identity/v3/endpoints?service_id=b6e9d59f4fa541028c59446dc7715f7c&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:47:09.203480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:09.283404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:09.300804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e6b6-8ae5-4699-8e89-f9e2f1005fa3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:09.332437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:09.333021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:09.333295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:09.333606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:09.334188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:09.335526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6722d-6a07-417a-a16a-b59fc1703fc4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:09.342594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 651/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Thu Aug 11 10:47:09 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:47:11.327241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fdb4-a9b4-4775-83be-24e158a4bd51 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:11.327806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fdb4-a9b4-4775-83be-24e158a4bd51 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:11.327806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6fdb4-a9b4-4775-83be-24e158a4bd51 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:11.329153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 651/1302] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:47:11.334538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:11.334924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:11.335079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:11.353127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:47:11.354288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:47:11.423845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 652/1303] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:47:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 10:47:11.451583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6fdb4-a9b4-4775-83be-24e158a4bd51 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:11.573102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:11.580900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6fdb4-a9b4-4775-83be-24e158a4bd51 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:11.626174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:11.626848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:11.627125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:11.627393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:11.628020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:11.630016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 10:47:11.642782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 10:47:11.669340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 11 10:47:11.670490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 652/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Thu Aug 11 10:47:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 10:47:11.671564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:11.741652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:11.765605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb203091-1eae-45d4-b7d1-c0f25ca68413 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:11.800140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:11.800738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:11.801015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:11.801284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:11.801905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:47:11.802202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:11.803550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4125ddc-50a1-411a-9ead-0b7cb31be34f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:11.809143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 653/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Thu Aug 11 10:47:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:47:11.815034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:47:11.918723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815df32-b269-4b8b-894a-d17689f45ae2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:47:11.952540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:47:11.953144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:47:11.959040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:47:11.959040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:47:11.959040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:47:11.959040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b680fb-2be9-4518-8aa8-a9e6b31c3a98 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:47:12.062503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 653/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 10:47:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:47:21.840367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:47:22.058001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:54:59.639695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a5fa-7b9b-4e3b-8fc9-1f6070a5a840 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:54:59.640293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a5fa-7b9b-4e3b-8fc9-1f6070a5a840 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:54:59.640293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156a5fa-7b9b-4e3b-8fc9-1f6070a5a840 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:54:59.646842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a5ba-b699-456f-af7b-188b0c4f9fdb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:54:59.647300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a5ba-b699-456f-af7b-188b0c4f9fdb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:54:59.647399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7a5ba-b699-456f-af7b-188b0c4f9fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:54:59.650865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 654/1307] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:54:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:54:59.739940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5cb7a5ba-b699-456f-af7b-188b0c4f9fdb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:54:59.741147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5cb7a5ba-b699-456f-af7b-188b0c4f9fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:54:59.938828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 654/1308] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:55:00.002030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:00.002314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:00.002623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:00.004211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 655/1309] 10.222.0.48 () {54 vars in 965 bytes} [Thu Aug 11 10:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:00.010366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf66ff-013f-4e49-8673-e2aca8698e6d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:00.010632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf66ff-013f-4e49-8673-e2aca8698e6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:00.011055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf66ff-013f-4e49-8673-e2aca8698e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:00.138938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-27cf66ff-013f-4e49-8673-e2aca8698e6d None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:00.140369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27cf66ff-013f-4e49-8673-e2aca8698e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:00.163696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:00.235751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 655/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Aug 11 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:00.246056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:00.247920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:00.378946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08399ba0-d8d2-49ea-a799-c7c747554887 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:00.478268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:00.478882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:00.479156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:00.479420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:00.481110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:00.542691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:00.609222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:00.611007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86411ea-2d53-4a59-a6e7-3470d4453dce service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:00.629986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 656/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:02.976605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:02.977173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:02.977173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:02.979183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 656/1312] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:02.984197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0108-40ad-495f-9b78-4f21985b5dd3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:02.984598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0108-40ad-495f-9b78-4f21985b5dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:02.984748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d0108-40ad-495f-9b78-4f21985b5dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:03.074661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d5d0108-40ad-495f-9b78-4f21985b5dd3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:03.075936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d5d0108-40ad-495f-9b78-4f21985b5dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:03.195067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 657/1313] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:03.233805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:03.235325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:03.339469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12bd39-a1ae-4622-8d39-28982d0e26bc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:03.412343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:03.415571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:03.416207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:03.416477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:03.416744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:03.418315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:03.530042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:03.531401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af11ae-99a4-42c9-9877-e09077ecf414 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:03.551041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 657/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:06.111493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:06.112138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:06.112138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:06.113436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 658/1315] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:06.119948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6104-104d-4d5d-80ef-b4fa3bc0921d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:06.120448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6104-104d-4d5d-80ef-b4fa3bc0921d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:06.120692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a6104-104d-4d5d-80ef-b4fa3bc0921d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:06.187013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5f3a6104-104d-4d5d-80ef-b4fa3bc0921d None None] MFA Rules not processed for user `ee0c0304aa16452fb5a1106b5a439277`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:06.188276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5f3a6104-104d-4d5d-80ef-b4fa3bc0921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:06.237857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:06.280605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 658/1316] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:06.319991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:06.322190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:06.331710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:06.422986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560e21-4d2b-414a-b9ad-4daa3c1d48a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:06.500336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:06.500942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:06.501221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:06.501493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:06.503088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:06.562669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:06.608358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:06.609784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7659ade-ec7f-4cb9-ae93-5decb6062ba4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:06.627754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 659/1317] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:09.484883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:09.485394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:09.485394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:09.486765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 659/1318] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:09.491753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942394-83bb-4e96-a562-35484d6e463f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:09.492212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942394-83bb-4e96-a562-35484d6e463f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:09.492316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13942394-83bb-4e96-a562-35484d6e463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:09.563063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-13942394-83bb-4e96-a562-35484d6e463f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:09.564294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13942394-83bb-4e96-a562-35484d6e463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:09.659101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 660/1319] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:09.698287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:09.699859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:09.801501 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51bab2-be13-4117-9904-1eb82fd5acc4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:09.873893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:09.874482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:09.874758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:09.875028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:09.876625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:09.883933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:09.983289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:09.984749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd9296-137c-4b39-88da-124c2a6909f1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:10.002842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 660/1320] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:12.919987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:12.920518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:12.920518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:12.921956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 661/1321] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:12.926862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5d3c-5963-4550-b255-bfaac72bd8a3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:12.927253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5d3c-5963-4550-b255-bfaac72bd8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:12.927421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c5d3c-5963-4550-b255-bfaac72bd8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:12.993805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7d0c5d3c-5963-4550-b255-bfaac72bd8a3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:12.995017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d0c5d3c-5963-4550-b255-bfaac72bd8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:13.051589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:13.091478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 661/1322] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:13.128135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:13.129669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:13.149508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:13.239444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b061b-cc05-4809-99f0-1f63235cd5d3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:13.318749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:13.319551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:13.319843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:13.320115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:13.322317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:13.396154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:13.434835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:13.436298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2094-f1c9-4204-ad76-371eb1ad425d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:13.453913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 662/1323] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:15.545003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:15.545548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:15.545548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:15.546939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 662/1324] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:15.552203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77cba-e155-4d5f-b7da-d0407934ae72 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:15.552631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77cba-e155-4d5f-b7da-d0407934ae72 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:15.552762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77cba-e155-4d5f-b7da-d0407934ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:15.622381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8db77cba-e155-4d5f-b7da-d0407934ae72 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:15.623612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8db77cba-e155-4d5f-b7da-d0407934ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:15.718241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 663/1325] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:15.754648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:15.756237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:15.858822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed6807-93a7-44ea-9566-3ba265450601 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:15.931182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:15.931794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:15.932074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:15.932339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:15.933957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:15.950882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:16.046512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:16.048164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f6347-d3f7-4ee7-ac1e-03f243f46aca service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:16.069128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 663/1326] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:18.114488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:18.115006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:18.115006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:18.116391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 664/1327] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:18.121373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908dd9-f622-42cc-a3d7-c365aa482a4f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:18.121785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908dd9-f622-42cc-a3d7-c365aa482a4f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:18.121922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc908dd9-f622-42cc-a3d7-c365aa482a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:18.192696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bc908dd9-f622-42cc-a3d7-c365aa482a4f None None] MFA Rules not processed for user `ee0c0304aa16452fb5a1106b5a439277`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:18.193886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc908dd9-f622-42cc-a3d7-c365aa482a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:18.259969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:18.290522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 664/1328] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:18.328462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:18.330116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:18.362462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:18.436601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3d148-7cbe-4c6e-b72e-908f666959ae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:18.512084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:18.512686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:18.512963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:18.513232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:18.514828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:18.602365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:18.635980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:18.637365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7b77c-340f-4253-af48-023c321da0d5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:18.655403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 665/1329] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:21.126277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:21.126797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:21.126797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:21.128113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 665/1330] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:21.133823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408745-4c80-4934-89aa-226796a0166e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:21.134235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408745-4c80-4934-89aa-226796a0166e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:21.134378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408745-4c80-4934-89aa-226796a0166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:21.198288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-42408745-4c80-4934-89aa-226796a0166e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:21.199490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42408745-4c80-4934-89aa-226796a0166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:21.296640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 666/1331] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:21.333575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:21.335099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:21.437612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bb751-00aa-40cf-a07b-103cf4d2a086 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:21.511338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:21.511971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:21.512246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:21.512513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:21.514090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:21.541199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:21.631892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:21.633380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b530-c601-43fe-9671-9189f88bdae2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:21.653282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 666/1332] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:23.762453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:23.763066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:23.763066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:23.764389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 667/1333] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:23.769415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2331-72ab-4fc6-b98e-cc1e7a7c6f13 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:23.769839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2331-72ab-4fc6-b98e-cc1e7a7c6f13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:23.769956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2331-72ab-4fc6-b98e-cc1e7a7c6f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:23.834290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-135d2331-72ab-4fc6-b98e-cc1e7a7c6f13 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:23.836029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-135d2331-72ab-4fc6-b98e-cc1e7a7c6f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:23.907707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:23.929636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 667/1334] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:23.970456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:23.972018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:24.011747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:24.078184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823b0e3-170b-4686-b57b-4edf4f020975 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:24.148694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:24.149288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:24.149564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:24.149829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:24.151393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:24.230703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:24.255307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:24.256738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde44446-a02c-42cd-b20d-7b30aba00f5a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:24.275981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 668/1335] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:26.899082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:26.899598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:26.899598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:26.900926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 668/1336] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:55:26.905991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe99e2-20e7-469e-a4e8-2965a548e6da None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:26.906387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe99e2-20e7-469e-a4e8-2965a548e6da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:26.906546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe99e2-20e7-469e-a4e8-2965a548e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:26.972268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fdbe99e2-20e7-469e-a4e8-2965a548e6da None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:26.973482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdbe99e2-20e7-469e-a4e8-2965a548e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:27.067754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 669/1337] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:27.104945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:27.106458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:27.206493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6ce62-d843-4fff-bf52-d550bd251f8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:27.277718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:27.278316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:27.278593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:27.278865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:27.280458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:27.314763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:27.386281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:27.387688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854efd2b-4343-4f39-a464-61d3a9c86124 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:27.405535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 669/1338] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:31.607646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:31.608073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:31.608238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:31.609616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 670/1339] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:31.615016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8171c0-b77a-4153-bb16-c579e01969d4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:31.615492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8171c0-b77a-4153-bb16-c579e01969d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:31.615591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8171c0-b77a-4153-bb16-c579e01969d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:31.688038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6a8171c0-b77a-4153-bb16-c579e01969d4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:31.689223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6a8171c0-b77a-4153-bb16-c579e01969d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:31.781145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:31.792293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 670/1340] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:31.832581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:31.834222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:31.884558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:31.942934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3a6ec-3503-4024-ba3a-3909aa932c87 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:32.015816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:32.016419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:32.016693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:32.016960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:32.018595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:32.113536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:32.129472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:32.130899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524f21d-ae51-420d-9573-dded12e86d25 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:32.148575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 671/1341] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:35.236957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a61a-8904-4f43-abee-e6b525920780 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:35.237522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a61a-8904-4f43-abee-e6b525920780 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:35.237628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942a61a-8904-4f43-abee-e6b525920780 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:35.238950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 671/1342] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:35.244119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96754-8693-4562-ba40-8fd8bf4e21ab None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:35.244600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96754-8693-4562-ba40-8fd8bf4e21ab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:35.244786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe96754-8693-4562-ba40-8fd8bf4e21ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:35.309402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5fe96754-8693-4562-ba40-8fd8bf4e21ab None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:35.310494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5fe96754-8693-4562-ba40-8fd8bf4e21ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:35.405777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 672/1343] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:35.457347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:35.457646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:35.457906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:35.459873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 672/1344] 10.222.0.48 () {54 vars in 965 bytes} [Thu Aug 11 10:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:35.465696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23eff-8339-4a0a-ba85-572f79a8f175 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:35.465977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23eff-8339-4a0a-ba85-572f79a8f175 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:35.466246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c23eff-8339-4a0a-ba85-572f79a8f175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:35.530699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:35.539367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d7c23eff-8339-4a0a-ba85-572f79a8f175 None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:35.540608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7c23eff-8339-4a0a-ba85-572f79a8f175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:35.619671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 673/1345] 10.222.0.48 () {58 vars in 1062 bytes} [Thu Aug 11 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:55:35.629244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:35.630770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:35.742037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172bae6-4cae-4d75-a3ab-d6c6a26d6b65 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:35.814704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:35.815351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:35.815628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:35.815938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:35.817515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:35.861612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:35.930287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:35.931729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18adea-62ca-44ec-bc7d-8e0fc4e6bafd service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:35.952951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 673/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:40.924895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:40.925429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:40.925429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:40.926795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 674/1347] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:40.932940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3aa7-ef2c-4813-a6ec-985d76c980f2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:40.933454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3aa7-ef2c-4813-a6ec-985d76c980f2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:40.933454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3aa7-ef2c-4813-a6ec-985d76c980f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:41.001381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3a8c3aa7-ef2c-4813-a6ec-985d76c980f2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:41.002654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a8c3aa7-ef2c-4813-a6ec-985d76c980f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:41.105763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:41.108506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 674/1348] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:41.147132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:41.148713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:41.254773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f02c-54fb-4410-8e5e-b04ac61c2148 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:41.301561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:41.328008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:41.328670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:41.328957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:41.329225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:41.330798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:41.449683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:41.451124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ace93-8d78-4473-b07d-dc4eab194a14 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:41.474139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 675/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:46.385246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290c-ec69-4bb5-9eed-4e30dc2cddc6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:46.385793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290c-ec69-4bb5-9eed-4e30dc2cddc6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:46.385793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b290c-ec69-4bb5-9eed-4e30dc2cddc6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:46.387378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 675/1350] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 10:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:46.392348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90abf-1e08-4879-8cab-e80509ddb97c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:46.392779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90abf-1e08-4879-8cab-e80509ddb97c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:46.392884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d90abf-1e08-4879-8cab-e80509ddb97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:46.516482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-91d90abf-1e08-4879-8cab-e80509ddb97c None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:46.517775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-91d90abf-1e08-4879-8cab-e80509ddb97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:46.553894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:46.680903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 676/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:50.723436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6080d16-ef8c-45d4-9b54-9892fe5b946f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:50.724099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6080d16-ef8c-45d4-9b54-9892fe5b946f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:50.724099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6080d16-ef8c-45d4-9b54-9892fe5b946f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:50.725739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 676/1352] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 10:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:50.731118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b926b51-4194-4940-9b7f-b1c132c35202 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:50.731596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b926b51-4194-4940-9b7f-b1c132c35202 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:50.731752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b926b51-4194-4940-9b7f-b1c132c35202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:50.812285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4b926b51-4194-4940-9b7f-b1c132c35202 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:50.813522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b926b51-4194-4940-9b7f-b1c132c35202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:50.887985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:50.925008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 677/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:52.425034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d330ac-648d-4a8f-8a92-d8b0daa5aec3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:52.425592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d330ac-648d-4a8f-8a92-d8b0daa5aec3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:52.425756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d330ac-648d-4a8f-8a92-d8b0daa5aec3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:52.427357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 677/1354] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:52.432662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19593d1-2396-4390-872f-36af66e2aa95 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:52.433098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19593d1-2396-4390-872f-36af66e2aa95 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:52.433290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19593d1-2396-4390-872f-36af66e2aa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:52.511542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e19593d1-2396-4390-872f-36af66e2aa95 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:52.512772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e19593d1-2396-4390-872f-36af66e2aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:52.622172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 678/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 10:55:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:55:56.052546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09139bd-9720-49a3-bfb4-2adc3af79c92 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:56.053140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09139bd-9720-49a3-bfb4-2adc3af79c92 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:56.053140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09139bd-9720-49a3-bfb4-2adc3af79c92 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:56.056780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 678/1356] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:55:56.063340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e441a-e3c1-44f9-b111-96ad7a8b8a89 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:56.063900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e441a-e3c1-44f9-b111-96ad7a8b8a89 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:56.064016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e441a-e3c1-44f9-b111-96ad7a8b8a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:56.110316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:56.148871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d0e441a-e3c1-44f9-b111-96ad7a8b8a89 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:56.150100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d0e441a-e3c1-44f9-b111-96ad7a8b8a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:56.257315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 679/1357] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:56.312011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:56.312522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:56.312955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:56.314451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 679/1358] 10.222.0.48 () {54 vars in 962 bytes} [Thu Aug 11 10:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:55:56.322387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf661-7e82-488e-8a08-eb5e3f2d1620 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:56.322788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf661-7e82-488e-8a08-eb5e3f2d1620 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:56.323068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf661-7e82-488e-8a08-eb5e3f2d1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:56.413537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3adcf661-7e82-488e-8a08-eb5e3f2d1620 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:56.414862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3adcf661-7e82-488e-8a08-eb5e3f2d1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:56.475348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:56.554190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 680/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Aug 11 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:56.561657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:56.563395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:56.715743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac4a0a-94eb-42ca-91f3-7fb0c6f9df61 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:56.789815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:56.830158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:56.830690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:56.830963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:56.831231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:56.833096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:56.924801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:56.926454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d523a79-35e8-4ab6-8bf0-ff9afb00bae2 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:56.947936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 680/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:55:59.154851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:59.155379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:59.155379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:59.156868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 681/1361] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:55:59.162264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d10ca-9f81-485d-a1c0-2e808029367c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:59.162688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d10ca-9f81-485d-a1c0-2e808029367c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:59.162897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d10ca-9f81-485d-a1c0-2e808029367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:59.232429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf9d10ca-9f81-485d-a1c0-2e808029367c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:55:59.233634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf9d10ca-9f81-485d-a1c0-2e808029367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:59.272665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:59.332956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 681/1362] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:55:59.376701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:55:59.378373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:59.541869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6328b6f-4b27-4692-b0cb-8907202c7adc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:55:59.556983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:55:59.672392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:55:59.672990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:55:59.673264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:55:59.673532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:55:59.675135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:55:59.797501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:55:59.799141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03e1a9-7827-472f-8a67-ff51398531d7 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:55:59.820832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 682/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:02.293929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770767-850d-4084-987e-83b1a65ee4c6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:02.294530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770767-850d-4084-987e-83b1a65ee4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:02.294629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a770767-850d-4084-987e-83b1a65ee4c6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:02.296185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 682/1364] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:02.301295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008ca78-cf30-47aa-9f5a-261d4576bde6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:02.301791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008ca78-cf30-47aa-9f5a-261d4576bde6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:02.302071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008ca78-cf30-47aa-9f5a-261d4576bde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:02.318603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:02.374918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6008ca78-cf30-47aa-9f5a-261d4576bde6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:02.376270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6008ca78-cf30-47aa-9f5a-261d4576bde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:02.491513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 683/1365] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:02.538647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:02.538935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:02.539214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:02.540750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 683/1366] 10.222.0.48 () {54 vars in 962 bytes} [Thu Aug 11 10:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:02.547322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019f13-be52-43fc-8c23-ba77b09cdabf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:02.547322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019f13-be52-43fc-8c23-ba77b09cdabf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:02.547322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50019f13-be52-43fc-8c23-ba77b09cdabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:02.640477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-50019f13-be52-43fc-8c23-ba77b09cdabf None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:02.641804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50019f13-be52-43fc-8c23-ba77b09cdabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:02.678376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:02.765264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 684/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Aug 11 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:02.773244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:02.775087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:02.942300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a36acc-0faa-4d20-aee3-f6f088b9e72d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:02.963710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:03.060477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:03.061241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:03.061620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:03.061991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:03.063732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:03.165288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:03.167127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7cb15-b341-4cb2-9f3f-c8007eace806 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:03.186950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 684/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:04.270236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58229-83d5-4656-b216-0b137bcb7487 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:04.270852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58229-83d5-4656-b216-0b137bcb7487 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:04.271072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58229-83d5-4656-b216-0b137bcb7487 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:04.272767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 685/1369] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:04.277743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46096826-cd18-406f-8fb2-ca0ad5021136 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:04.278171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46096826-cd18-406f-8fb2-ca0ad5021136 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:04.278500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46096826-cd18-406f-8fb2-ca0ad5021136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:04.310931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:04.345215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-46096826-cd18-406f-8fb2-ca0ad5021136 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:04.346426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-46096826-cd18-406f-8fb2-ca0ad5021136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:04.447136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 685/1370] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:04.813579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb649f-6253-435a-b398-87fc805df59f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:04.813863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb649f-6253-435a-b398-87fc805df59f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:04.814166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb649f-6253-435a-b398-87fc805df59f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:04.815664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 686/1371] 10.222.0.48 () {54 vars in 964 bytes} [Thu Aug 11 10:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:56:04.821393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473a82-c19f-409a-b657-5b5632aa4c26 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:04.821863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473a82-c19f-409a-b657-5b5632aa4c26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:04.822306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473a82-c19f-409a-b657-5b5632aa4c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:04.905809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01473a82-c19f-409a-b657-5b5632aa4c26 None None] MFA Rules not processed for user `a1e9a4ceebcc44389144522cec3cc678`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:04.906934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01473a82-c19f-409a-b657-5b5632aa4c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:04.957955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:04.986975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 686/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Aug 11 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:04.995077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb649f-6253-435a-b398-87fc805df59f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:04.996892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edeb649f-6253-435a-b398-87fc805df59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:05.118173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb649f-6253-435a-b398-87fc805df59f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:05.161593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:05.198138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:05.198890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:05.199269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:05.199646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:05.201561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:05.320460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:05.322080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cc067-d1c0-4c38-8ad1-bc66ed836094 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:05.343417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 687/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:05.426217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:05.426786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:05.426786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:05.431920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 687/1374] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:05.433665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8eb3b-9122-4eee-8c41-1701ad5a7e02 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:05.433665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8eb3b-9122-4eee-8c41-1701ad5a7e02 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:05.433977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8eb3b-9122-4eee-8c41-1701ad5a7e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:05.501678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-37d8eb3b-9122-4eee-8c41-1701ad5a7e02 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:05.502886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37d8eb3b-9122-4eee-8c41-1701ad5a7e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:05.524345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:05.610580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 688/1375] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:56:05.655532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:05.657069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:05.818923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2d0a1-aa41-406b-8d07-1caaca10c7ac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:05.871081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:05.933596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:05.934112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:05.934393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:05.934802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:05.936470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:06.026182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:06.028619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b3e5-fa93-4ce4-85e9-c11090b4d25f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:06.047232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 688/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:08.197372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:08.197897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:08.198075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:08.199579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 689/1377] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:08.204997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825d7a-3dcc-442c-9974-67f8880d1a18 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:08.205412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825d7a-3dcc-442c-9974-67f8880d1a18 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:08.205556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825d7a-3dcc-442c-9974-67f8880d1a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:08.271907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c3825d7a-3dcc-442c-9974-67f8880d1a18 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:08.273117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3825d7a-3dcc-442c-9974-67f8880d1a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:08.279623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:08.369945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 689/1378] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:08.417123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:08.418639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:08.592102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868f06-d77c-40da-9208-8578069587f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:08.595909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:08.709591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:08.710197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:08.710475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:08.710747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:08.712357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:08.804777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:08.806408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f67ab5-991d-41e2-8b98-4240dab865a1 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:08.824858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 690/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:11.009071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:11.009667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:11.009667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:11.011268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 690/1380] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:11.016846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ab5e0-90c9-4907-8297-26e250dc9449 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:11.017423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ab5e0-90c9-4907-8297-26e250dc9449 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:11.017423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ab5e0-90c9-4907-8297-26e250dc9449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:11.030965 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:11.216617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c00ab5e0-90c9-4907-8297-26e250dc9449 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:11.217885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c00ab5e0-90c9-4907-8297-26e250dc9449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:11.317949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 691/1381] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:11.361950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:11.363714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:11.509901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:11.520614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e74a-0376-4ca7-9756-1e8f7e18c020 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:11.651748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:11.652659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:11.652937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:11.653206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:11.654784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:11.750272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:11.751887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520eac2-56bf-467e-988f-dd695e34b7af service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:11.763075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:11.778526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 691/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:14.025664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:14.026585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:14.026907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:14.028310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 692/1383] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:14.034935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570576-58b1-4959-9d46-0db43c2a6db9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:14.035371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570576-58b1-4959-9d46-0db43c2a6db9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:14.035861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570576-58b1-4959-9d46-0db43c2a6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:14.115035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d8570576-58b1-4959-9d46-0db43c2a6db9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:14.116653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d8570576-58b1-4959-9d46-0db43c2a6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:14.218276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 692/1384] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:14.263305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:14.265294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:14.365002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:14.430618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a2e77-9480-403f-98fb-92684d80a758 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:14.542931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:14.543554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:14.543866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:14.544133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:14.545805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:14.600875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:14.645414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:14.646931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31733d9-4489-4b92-aab9-207257ab8ae9 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:14.665298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 693/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:17.145886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:17.146545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:17.146545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:17.147859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 693/1386] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:17.153115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f041-db25-44e9-ac3f-88c4b259805e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:17.153689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f041-db25-44e9-ac3f-88c4b259805e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:17.153689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0f041-db25-44e9-ac3f-88c4b259805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:17.221308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cca0f041-db25-44e9-ac3f-88c4b259805e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:17.223085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cca0f041-db25-44e9-ac3f-88c4b259805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:17.324741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 694/1387] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:17.369766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:17.371305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:17.450248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:17.524751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8cef0-0cb1-463d-a968-df2a25fd09f2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:17.645944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:17.646561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:17.646841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:17.647110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:17.648774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:17.692078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:17.748068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:17.749740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f679-507a-47f4-91b7-c0314b959997 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:17.769335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 694/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:19.967375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:19.967975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:19.967975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:19.969353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 695/1389] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:19.975329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007e1f-880a-46f2-bbd4-12e082546605 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:19.978096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007e1f-880a-46f2-bbd4-12e082546605 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:19.978455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d007e1f-880a-46f2-bbd4-12e082546605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:20.058208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0d007e1f-880a-46f2-bbd4-12e082546605 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:20.059456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d007e1f-880a-46f2-bbd4-12e082546605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:20.159826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 695/1390] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:20.203912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:20.205841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:20.237096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:20.357628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abbd44-fcdb-46ad-8645-4f27a9c4e651 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:20.458934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:20.470737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:20.471343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:20.471621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:20.471931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:20.473511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:20.564455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:20.566471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e8c5f-6132-4b79-aaa4-feed82dc810c service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:20.584455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 696/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:22.030486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf3b5-24a4-43e8-9876-8137189f41b9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.031068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf3b5-24a4-43e8-9876-8137189f41b9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.031068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf3b5-24a4-43e8-9876-8137189f41b9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.032445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 696/1392] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:22.038017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63145664-ac3b-411d-b031-41e1a142ee0b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.038678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63145664-ac3b-411d-b031-41e1a142ee0b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.038678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63145664-ac3b-411d-b031-41e1a142ee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.108382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-63145664-ac3b-411d-b031-41e1a142ee0b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:22.109592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63145664-ac3b-411d-b031-41e1a142ee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:22.167551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:22.205687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 697/1393] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:22.262171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.262456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.262727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.264121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 697/1394] 10.222.0.48 () {54 vars in 967 bytes} [Thu Aug 11 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:22.269127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4a39-ffd7-47c4-8044-f4e9528ba63d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.269426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4a39-ffd7-47c4-8044-f4e9528ba63d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.269697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4a39-ffd7-47c4-8044-f4e9528ba63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.350927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f11f4a39-ffd7-47c4-8044-f4e9528ba63d None None] MFA Rules not processed for user `a947c2e6ba544767897eac257c1fad7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:22.352217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f11f4a39-ffd7-47c4-8044-f4e9528ba63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:22.433331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 698/1395] 10.222.0.48 () {58 vars in 1064 bytes} [Thu Aug 11 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:22.441690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:22.443200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:22.458218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:22.550611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf980a-1d5a-480f-bda9-b066e5278c8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:22.629963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:22.630566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.630841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.631112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.632786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:22.705906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:22.750168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:22.751620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65b1a5-6ef3-4f49-add4-9bc09812a3a2 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:22.770298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 698/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:22.849996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.850309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.850594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.852090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 699/1397] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:22.857745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477632-2310-457a-8aee-41bb25c56b9f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:22.857979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477632-2310-457a-8aee-41bb25c56b9f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:22.858257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477632-2310-457a-8aee-41bb25c56b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:22.926726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e2477632-2310-457a-8aee-41bb25c56b9f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:22.928069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2477632-2310-457a-8aee-41bb25c56b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:23.031200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 699/1398] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:23.076229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:23.077893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:23.144080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:23.235585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711d1e7-7cc0-46a2-8abe-a84901416692 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:23.353083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:23.353692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:23.353970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:23.354236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:23.355850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:23.376698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:23.458368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:23.460339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711325c-1a64-46d6-8953-e0f7ebd3fc51 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:23.480508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 700/1399] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:23.645042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f284f-a4ed-49f8-b7f3-e1367f9ee5dd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:23.650253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f284f-a4ed-49f8-b7f3-e1367f9ee5dd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:23.650253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f284f-a4ed-49f8-b7f3-e1367f9ee5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:23.671319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:23.719469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-704f284f-a4ed-49f8-b7f3-e1367f9ee5dd None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:23.720942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-704f284f-a4ed-49f8-b7f3-e1367f9ee5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:23.829140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 700/1400] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:23.878642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:23.879383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:23.879863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:23.881563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 701/1401] 10.222.0.48 () {54 vars in 943 bytes} [Thu Aug 11 10:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:23.889546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380ecd-ff6a-4051-918b-531ae0c19899 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:23.890062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380ecd-ff6a-4051-918b-531ae0c19899 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:23.890496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380ecd-ff6a-4051-918b-531ae0c19899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:23.979125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-97380ecd-ff6a-4051-918b-531ae0c19899 None None] MFA Rules not processed for user `d0dea5e1e47b48b0b7623c6b7100279e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:23.980419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-97380ecd-ff6a-4051-918b-531ae0c19899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:24.040725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:24.112792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 701/1402] 10.222.0.48 () {58 vars in 1040 bytes} [Thu Aug 11 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:24.122294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:24.123846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:24.265403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423430c-c17f-499e-8a28-babdbeb679e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:24.277858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:24.368229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:24.368854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:24.369127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:24.369393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:24.370984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:24.460975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:24.462539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a10fa-3d94-45ac-bab4-8199c5eb7568 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:24.470676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 702/1403] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 10:56:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:56:25.817775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:25.818391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:25.818696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:25.824850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 702/1404] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:25.825680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04cb3-cf26-46ef-9855-7684f8e28b91 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:25.826283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04cb3-cf26-46ef-9855-7684f8e28b91 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:25.826566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04cb3-cf26-46ef-9855-7684f8e28b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:25.886649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:25.900502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-52d04cb3-cf26-46ef-9855-7684f8e28b91 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:25.901714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-52d04cb3-cf26-46ef-9855-7684f8e28b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:26.016126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 703/1405] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:56:26.056398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:26.058089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:26.220432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ada69-a66b-46d4-91c0-10df903713dc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:26.250226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:26.345210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:26.345835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:26.346109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:26.346376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:26.348018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:26.449589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:26.451719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57886653-c9f7-4f86-823a-7185fde0c9e1 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:26.471452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 703/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:28.657972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:28.658545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:28.658545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:28.663628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 704/1407] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:56:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:28.665089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc6e0-054f-4a05-847b-3257138295bc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:28.665527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc6e0-054f-4a05-847b-3257138295bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:28.665670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cc6e0-054f-4a05-847b-3257138295bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:28.719025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:28.749782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-384cc6e0-054f-4a05-847b-3257138295bc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:28.751196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-384cc6e0-054f-4a05-847b-3257138295bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:28.866086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 704/1408] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:56:28.910920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:28.912535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:29.054696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:29.086278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e605a11-dae8-4f8c-82eb-de43f24f669c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:29.213264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:29.214232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:29.214717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:29.214985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:29.216614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:29.301581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:29.311254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:29.314826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb541-65c0-4c6e-9689-ca63594dec3e service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:29.340097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 705/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:38.864251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=51 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:39.338431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:47.790453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760caf-2643-4718-9d51-6bf55635bddf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:47.790993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760caf-2643-4718-9d51-6bf55635bddf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:47.790993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760caf-2643-4718-9d51-6bf55635bddf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:47.792331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 705/1410] 10.222.0.48 () {54 vars in 918 bytes} [Thu Aug 11 10:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:47.797403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cfa9d-cc92-4b93-bba4-ed9f37fdb814 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:47.797835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cfa9d-cc92-4b93-bba4-ed9f37fdb814 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:47.797941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cfa9d-cc92-4b93-bba4-ed9f37fdb814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:47.873333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-013cfa9d-cc92-4b93-bba4-ed9f37fdb814 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:47.874510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-013cfa9d-cc92-4b93-bba4-ed9f37fdb814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:47.978758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 706/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Aug 11 10:56:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:47.993160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81a94-a679-4c62-a137-18e731313a76 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:47.993451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81a94-a679-4c62-a137-18e731313a76 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:47.993759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b81a94-a679-4c62-a137-18e731313a76 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:47.995083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 706/1412] 10.222.0.48 () {54 vars in 918 bytes} [Thu Aug 11 10:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:47.999628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4423-2242-4b2e-9ace-ff0cab780813 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:47.999948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4423-2242-4b2e-9ace-ff0cab780813 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:48.000716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4423-2242-4b2e-9ace-ff0cab780813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:48.074719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:48.077435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-21eb4423-2242-4b2e-9ace-ff0cab780813 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:48.079251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21eb4423-2242-4b2e-9ace-ff0cab780813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:48.193997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 707/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Thu Aug 11 10:56:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:50.712646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:50.713227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:50.713227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:50.714554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:50.720250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d4a7-230e-4fd4-b887-7f133e22c6a8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:50.720652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d4a7-230e-4fd4-b887-7f133e22c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:50.720801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d4a7-230e-4fd4-b887-7f133e22c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:50.784667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dbe7d4a7-230e-4fd4-b887-7f133e22c6a8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:50.785862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbe7d4a7-230e-4fd4-b887-7f133e22c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:50.838602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:50.879490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:50.922442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:50.923988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:51.066289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8fd2d-e3a9-47f3-8c52-9cfee1f8ce1d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:51.136878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:51.176431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:51.177024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:51.177298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:51.177564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:51.179147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:51.266169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:51.267743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb3370-ff77-4835-be0d-c5c63cb7177c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:51.285708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 708/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:52.167587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:52.168171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:52.168171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:52.169510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:52.174800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360fd2-8fae-4591-b9b3-2424188c17fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:52.175277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360fd2-8fae-4591-b9b3-2424188c17fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:52.175381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360fd2-8fae-4591-b9b3-2424188c17fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:52.251730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-17360fd2-8fae-4591-b9b3-2424188c17fe None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:52.252954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-17360fd2-8fae-4591-b9b3-2424188c17fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:52.279246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:52.348309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:52.497699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:52.499215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:52.641130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a57684-a3e7-43a7-89c0-8e0c02a45dbc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:52.680109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:52.752448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:52.752969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:52.753245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:52.753514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:52.755099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:52.841783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:52.843386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd2fc0-6c99-47d5-910e-07947ae4b2da service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:52.861388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 710/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:53.736879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910206-67ce-4260-90e6-4da4500395db None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:53.737408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910206-67ce-4260-90e6-4da4500395db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:53.737408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39910206-67ce-4260-90e6-4da4500395db None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:53.738691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:53.743854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6868b-1490-4be0-b005-a422b1ec945d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:53.744317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6868b-1490-4be0-b005-a422b1ec945d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:53.744408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6868b-1490-4be0-b005-a422b1ec945d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:53.801809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:53.825282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-36b6868b-1490-4be0-b005-a422b1ec945d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:53.826711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-36b6868b-1490-4be0-b005-a422b1ec945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:53.930217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:54.087099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39910206-67ce-4260-90e6-4da4500395db None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:54.088572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39910206-67ce-4260-90e6-4da4500395db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:54.230335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39910206-67ce-4260-90e6-4da4500395db None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:54.242977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:54.342686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:54.343202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:54.343507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:54.343778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:54.345362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:54.430872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:54.432478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce857a-acc3-4396-b92f-23891f77b3d3 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:54.450217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 711/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:55.330222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:55.330777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:55.330777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:55.332167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:55.336967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717af0b8-b1a1-41ee-b064-54e308a9e106 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:55.337385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717af0b8-b1a1-41ee-b064-54e308a9e106 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:55.337524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717af0b8-b1a1-41ee-b064-54e308a9e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:55.357621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:55.413430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-717af0b8-b1a1-41ee-b064-54e308a9e106 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:55.414632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-717af0b8-b1a1-41ee-b064-54e308a9e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:55.512741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:55.663432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:55.664993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:55.773772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:55.814867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742843b7-f34d-431b-937a-7f9408358c4d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:55.924608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:55.925120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:55.925395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:55.925662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:55.927232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:56.000711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:56.013584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:56.015633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b2639-c0de-4be2-a3fb-45ec015934ed service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:56.033403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 713/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:56.907158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:56.907741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:56.907741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:56.909073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:56.913855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18540b-d7a8-4bd3-a6f9-290a1e10533a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:56.914263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18540b-d7a8-4bd3-a6f9-290a1e10533a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:56.914373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18540b-d7a8-4bd3-a6f9-290a1e10533a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:56.987666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aa18540b-d7a8-4bd3-a6f9-290a1e10533a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:56.988878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa18540b-d7a8-4bd3-a6f9-290a1e10533a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:57.081837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:57.230090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:57.231589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:57.313366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:57.373245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc34530-d9ab-4f15-b9b0-dcb34ecf6de9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:57.486690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:57.487246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:57.487500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:57.487787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:57.489337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:57.535918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:57.574974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:57.576592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c25011-5565-45e6-85ac-995cc16647c0 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:57.595533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 714/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:56:58.473684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:58.474294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:58.474294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:58.475569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:56:58.480572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3dd1-a596-418b-aa22-8799ed7230d8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:58.481032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3dd1-a596-418b-aa22-8799ed7230d8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:58.481176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d3dd1-a596-418b-aa22-8799ed7230d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:58.557442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c34d3dd1-a596-418b-aa22-8799ed7230d8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:56:58.558725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c34d3dd1-a596-418b-aa22-8799ed7230d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:58.657245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:56:58.808544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:56:58.809979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:58.858380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:58.952936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d67f52-7d39-4b36-afb9-b70913a19f5b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:56:59.060860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:56:59.061387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:56:59.062208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:56:59.062491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:56:59.064138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:56:59.070947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:56:59.150858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:56:59.152483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa768445-6335-497e-b580-53d8faa492b7 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:56:59.170423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 716/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:00.031406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965884-758d-45d1-93ed-2c83057da60f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:00.031985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965884-758d-45d1-93ed-2c83057da60f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:00.031985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965884-758d-45d1-93ed-2c83057da60f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:00.033480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:00.038647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2791-bb38-4e84-a55e-7e6bb5c9f829 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:00.039076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2791-bb38-4e84-a55e-7e6bb5c9f829 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:00.039183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2791-bb38-4e84-a55e-7e6bb5c9f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:00.114234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8c9e2791-bb38-4e84-a55e-7e6bb5c9f829 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:00.115451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8c9e2791-bb38-4e84-a55e-7e6bb5c9f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:00.183040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:00.212012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:00.361673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965884-758d-45d1-93ed-2c83057da60f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:00.363098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84965884-758d-45d1-93ed-2c83057da60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:00.380726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:00.504366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965884-758d-45d1-93ed-2c83057da60f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:00.588424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:00.614239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:00.614785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:00.615058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:00.615324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:00.617643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:00.705180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:00.706782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc1f7c-ca74-435e-9811-f3e51eec0cef service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:00.724876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 717/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:01.588705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:01.589257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:01.589257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:01.590611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:01.596311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793da9d-0130-4965-b3cf-b570c0f9e1e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:01.596755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793da9d-0130-4965-b3cf-b570c0f9e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:01.596863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793da9d-0130-4965-b3cf-b570c0f9e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:01.672074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a793da9d-0130-4965-b3cf-b570c0f9e1e5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:01.673869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a793da9d-0130-4965-b3cf-b570c0f9e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:01.715861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:01.770856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:01.925461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:01.927020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:02.069301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d5f68-e9c5-4399-9a46-9d20cbd5922e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:02.122253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:02.180593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:02.181181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:02.181810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:02.181810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:02.183299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:02.273025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:02.273354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ce526-0d35-4ebf-b951-748ef7b31952 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:02.292151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 719/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:03.217239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:03.217807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:03.217807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:03.219112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 10:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:03.224726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8112e6-68ee-4143-9d05-1fe73188448f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:03.225134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8112e6-68ee-4143-9d05-1fe73188448f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:03.225304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8112e6-68ee-4143-9d05-1fe73188448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:03.298988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8d8112e6-68ee-4143-9d05-1fe73188448f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:03.300210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d8112e6-68ee-4143-9d05-1fe73188448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:03.302176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:03.394985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:03.544674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:03.546196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:03.689090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4fd4f-a357-48a5-9f6e-1c1caa6ae4b0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:03.716161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:03.800459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:03.800995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:03.801273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:03.801546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:03.803145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:03.892108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:03.893741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158f9dc-0772-4d9e-93f8-e4ef7c2f933c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:03.911781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 720/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:05.848975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:05.849583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:05.849583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:05.850850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:05.857366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa8cb-0c3d-460d-ad61-b144849d8d8f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:05.857776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa8cb-0c3d-460d-ad61-b144849d8d8f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:05.857885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fa8cb-0c3d-460d-ad61-b144849d8d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:05.897352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:05.923080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a51fa8cb-0c3d-460d-ad61-b144849d8d8f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:05.924940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a51fa8cb-0c3d-460d-ad61-b144849d8d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:06.018539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:06.057599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:06.059221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:06.162499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ea22-d539-4d6d-91dc-3d723a0bf9a1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:06.194074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:06.234758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:06.235344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:06.235616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:06.235913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:06.237472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:06.341507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:06.342837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043ef8b-0707-461d-9a6b-963a4daeb46c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:06.361893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 722/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:08.512219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:08.512843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:08.512843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:08.514100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:08.519774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92f3ef-fe45-473a-99b1-7bde7c29ca82 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:08.520177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92f3ef-fe45-473a-99b1-7bde7c29ca82 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:08.520334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92f3ef-fe45-473a-99b1-7bde7c29ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:08.585032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a92f3ef-fe45-473a-99b1-7bde7c29ca82 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:08.586251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a92f3ef-fe45-473a-99b1-7bde7c29ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:08.599696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:08.682518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:08.721786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:08.723413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:08.824642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8226dfb-a613-4fac-8486-ff7960fa8cc1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:08.837221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:08.897437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:08.898035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:08.898312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:08.898581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:08.900202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:09.005830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:09.007288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe109b2d-da3a-43f2-a758-4c44342696d6 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:09.025461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 723/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:11.374631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:11.375713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:11.375713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:11.377063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:11.382179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9749f-22cb-4b2c-a7f6-1685dc3991aa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:11.382642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9749f-22cb-4b2c-a7f6-1685dc3991aa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:11.382752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9749f-22cb-4b2c-a7f6-1685dc3991aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:11.430262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:11.447818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6be9749f-22cb-4b2c-a7f6-1685dc3991aa None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:11.449009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6be9749f-22cb-4b2c-a7f6-1685dc3991aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:11.548295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:57:11.587742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:11.589292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:11.691031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89582f62-d316-4fdf-9e1c-4892f25f725e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:11.729985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:11.763802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:11.764409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:11.764685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:11.764952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:11.766560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:11.879351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:11.880816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651d24-f851-4f79-bb03-d0aab673d7f2 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:11.898843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 725/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:57:14.011188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:14.011787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:14.011787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:14.013152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:14.018801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5325-e5ee-4728-8093-58368a79f51d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:14.019257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5325-e5ee-4728-8093-58368a79f51d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:14.019373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf5325-e5ee-4728-8093-58368a79f51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:14.084120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eadf5325-e5ee-4728-8093-58368a79f51d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:14.085319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eadf5325-e5ee-4728-8093-58368a79f51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:14.105437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:14.178785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:14.217987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:14.219523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:14.319914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53a1fb-a9cf-401f-8bc7-cc09680a6400 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:14.339915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:14.392742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:14.393338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:14.393615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:14.393885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:14.395683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:14.502886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:14.504345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763170e9-d319-4f18-8122-a3f45cd98e33 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:14.521719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 726/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:16.583133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:16.583712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:16.583712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:16.585588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:57:16.591193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847d80-57ae-493d-8351-c16cc820c45b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:16.593631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847d80-57ae-493d-8351-c16cc820c45b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:16.593631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847d80-57ae-493d-8351-c16cc820c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:16.654763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:16.657402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dd847d80-57ae-493d-8351-c16cc820c45b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:16.658595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd847d80-57ae-493d-8351-c16cc820c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:16.755379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:57:16.793364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:16.794887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:16.897106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515c86-bc61-4bf6-912a-80031c32d181 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:16.944843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:16.968738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:16.969325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:16.969598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:16.969862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:16.971409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:17.076037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:17.077463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec114a-9fda-451f-a2da-3c2911d3dcf7 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:17.096083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 728/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:57:19.196189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:19.196784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:19.196784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:19.198076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:19.203673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abe0f-d36e-4f3c-95a4-b5e114fb3c6d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:19.204120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abe0f-d36e-4f3c-95a4-b5e114fb3c6d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:19.204242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2abe0f-d36e-4f3c-95a4-b5e114fb3c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:19.269100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f2abe0f-d36e-4f3c-95a4-b5e114fb3c6d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:19.270300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f2abe0f-d36e-4f3c-95a4-b5e114fb3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:19.298995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:19.364698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:19.402309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:19.406209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:19.507689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab926062-599c-48e6-af2e-4dd8daf54ea8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:19.654564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:19.699582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:19.700206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:19.700479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:19.700743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:19.702840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:19.806109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:19.807454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39348a7-670d-48e0-a4ce-679a210135b0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:19.824569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 729/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:21.873204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:21.873740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:21.873740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:21.875572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:21.880663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e364-4cd7-4414-96b4-1bcc37388fee None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:21.881124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e364-4cd7-4414-96b4-1bcc37388fee None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:21.881229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e364-4cd7-4414-96b4-1bcc37388fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:21.945001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8de7e364-4cd7-4414-96b4-1bcc37388fee None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:21.946165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8de7e364-4cd7-4414-96b4-1bcc37388fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:21.954340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:22.038055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:22.075916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:22.077483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:22.179318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577fa25-2f65-4c97-a8f2-51208e11753e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:22.232870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:22.251729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:22.252354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:22.252629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:22.252895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:22.254458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:22.361860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:22.363369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72256695-91fc-4552-a843-785ca01d1e95 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:22.382792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 731/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:24.416379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:24.416974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:24.416974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:24.418269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:24.423509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4656dc-3d0e-4a6a-b755-e316a1b115fb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:24.423916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4656dc-3d0e-4a6a-b755-e316a1b115fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:24.424110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4656dc-3d0e-4a6a-b755-e316a1b115fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:24.488982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ef4656dc-3d0e-4a6a-b755-e316a1b115fb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:24.490187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef4656dc-3d0e-4a6a-b755-e316a1b115fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:24.529725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:24.587798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:24.625018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:24.626584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:24.726059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066c5b9-efa6-42a8-98ce-47f271f948e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:24.762166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:24.798503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:24.799099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:24.799374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:24.799643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:24.801245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:24.907010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:24.908452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5bb87-8c2f-4893-871a-d27e4ab8cc1f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:24.927185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 732/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:26.958881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:26.959426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:26.959426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:26.960984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:26.966414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214134b2-106e-43d9-a0f6-50d079171897 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:26.966890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214134b2-106e-43d9-a0f6-50d079171897 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:26.967000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214134b2-106e-43d9-a0f6-50d079171897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:27.031185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-214134b2-106e-43d9-a0f6-50d079171897 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:27.032399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-214134b2-106e-43d9-a0f6-50d079171897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:27.050414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:27.127736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:27.164040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:27.165562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:27.275035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c23c-8f29-44a6-b0ea-21b7c04611e7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:27.343662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:27.350282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:27.350923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:27.351199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:27.351460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:27.353054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:27.458414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:27.459846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae167e1-bcda-47d6-b06c-dd15376c7eca service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:27.477609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 734/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:29.555500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:29.556065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:29.556065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:29.557395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:29.562737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8f83-f17b-4eee-8348-4dd4ff21c4da None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:29.563183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8f83-f17b-4eee-8348-4dd4ff21c4da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:29.563278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8f83-f17b-4eee-8348-4dd4ff21c4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:29.628342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-50ce8f83-f17b-4eee-8348-4dd4ff21c4da None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:29.629988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50ce8f83-f17b-4eee-8348-4dd4ff21c4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:29.676243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:29.723619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:29.760236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:29.761785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:29.862912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db35e4-48f5-43ba-9ce7-c88f3852b99e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:29.909081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:29.933960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:29.934562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:29.934839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:29.935110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:29.937274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:30.044914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:30.046338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c7f6-7704-4c0d-8ba1-42fe6b2d1beb service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:30.065056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 735/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:39.721828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=22 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:40.062839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=66 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:49.525325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882408e8-297b-482d-a534-6b13051dacac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:49.525846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882408e8-297b-482d-a534-6b13051dacac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:49.525846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882408e8-297b-482d-a534-6b13051dacac None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:49.527217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 736/1471] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:49.532829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b73f-b00f-447e-904a-c1c0c39f8e4b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:49.533322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b73f-b00f-447e-904a-c1c0c39f8e4b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:49.533436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403b73f-b00f-447e-904a-c1c0c39f8e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:49.598360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e403b73f-b00f-447e-904a-c1c0c39f8e4b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:49.601460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e403b73f-b00f-447e-904a-c1c0c39f8e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:49.695394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 736/1472] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:49.779546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:49.779861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:49.780133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:49.781462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 737/1473] 10.222.0.48 () {54 vars in 967 bytes} [Thu Aug 11 10:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:49.786301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2ddb-c158-47af-8595-783f662d2293 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:49.786588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2ddb-c158-47af-8595-783f662d2293 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:49.786860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e2ddb-c158-47af-8595-783f662d2293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:49.861510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-286e2ddb-c158-47af-8595-783f662d2293 None None] MFA Rules not processed for user `a947c2e6ba544767897eac257c1fad7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:49.862700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-286e2ddb-c158-47af-8595-783f662d2293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:49.883037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:49.937682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 737/1474] 10.222.0.48 () {58 vars in 1064 bytes} [Thu Aug 11 10:57:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:49.945204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:49.946716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:50.048988 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92237feb-50b8-4cac-95ff-908f6c626a8f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:50.121454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:50.122041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:50.122314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:50.122579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:50.124170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:50.177935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:50.232428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:50.233847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d309c-406a-474f-b3ef-3823699d3fbf service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:50.252342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 738/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:52.330492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:52.331068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:52.331068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:52.332528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 738/1476] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:52.338032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36cc4-bd10-4ef4-bfb9-66d7bc938b22 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:52.338515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36cc4-bd10-4ef4-bfb9-66d7bc938b22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:52.338614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc36cc4-bd10-4ef4-bfb9-66d7bc938b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:52.404908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-afc36cc4-bd10-4ef4-bfb9-66d7bc938b22 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:52.406104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-afc36cc4-bd10-4ef4-bfb9-66d7bc938b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:52.502268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 739/1477] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:52.525545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:52.527062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:52.654297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a2ce1-af62-476f-ae6f-aabba2b481d0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:52.701573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:52.744379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:52.744976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:52.745251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:52.745518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:52.746798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:52.749447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 10:57:52.749447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 10:57:52.749447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 10:57:52.751180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2452e2-f4c9-420e-9106-f221e771fa53 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:52.769188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 739/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Thu Aug 11 10:57:52 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:57:54.792025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:54.792594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:54.792703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:54.794544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 740/1479] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:57:54.799798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f64d07-015c-46b5-8c54-5245c1e8f92b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:54.800233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f64d07-015c-46b5-8c54-5245c1e8f92b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:54.800362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f64d07-015c-46b5-8c54-5245c1e8f92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:54.864407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-19f64d07-015c-46b5-8c54-5245c1e8f92b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:54.865513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-19f64d07-015c-46b5-8c54-5245c1e8f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:54.959215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 740/1480] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:55.001158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:55.002673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:55.013093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:55.146269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a37bc-b7ac-4e4b-ac9f-991b6d2d1ba5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:55.224304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:55.255525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:55.256163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:55.256441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:55.256710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:55.258259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:55.345855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:55.347409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4ffa2-b6c9-49b4-9d33-4bede126306c service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:55.364689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 741/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:57:57.549732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:57.550223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:57.550335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:57.551648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 741/1482] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:57:57.556742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c080-7092-4574-9a11-d4282a425cca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:57.557183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c080-7092-4574-9a11-d4282a425cca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:57.557282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c080-7092-4574-9a11-d4282a425cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:57.622869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c204c080-7092-4574-9a11-d4282a425cca None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:57:57.624085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c204c080-7092-4574-9a11-d4282a425cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:57.669513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:57.722307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 742/1483] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:57:57.761202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:57:57.762672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:57.785944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:57.865691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149f507-7cd0-4a16-a9ee-38e1af26e394 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:57:57.936488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:57:57.937084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:57:57.937356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:57:57.937623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:57:57.939177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:57:58.013016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:57:58.050777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:57:58.053587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de2b76-764e-47da-b95b-2bea9a74157a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:57:58.072856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 742/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:58:00.138075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:00.139254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:00.139254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:00.140092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 743/1485] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:58:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:58:00.145186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5713e-7555-4a0f-8c02-a442b9670256 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:00.145636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5713e-7555-4a0f-8c02-a442b9670256 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:00.145744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5713e-7555-4a0f-8c02-a442b9670256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:00.210198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e8a5713e-7555-4a0f-8c02-a442b9670256 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:58:00.211385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8a5713e-7555-4a0f-8c02-a442b9670256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:00.305132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 743/1486] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:58:00.347945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:58:00.349474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:00.451556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95866aea-4376-430d-8157-a98ea4e0723e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:58:00.523561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:00.525424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:58:00.526056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:00.526337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:00.527131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:00.528790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:00.636583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:58:00.638010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27a5d7-9f35-47ac-b999-b8772932a190 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:58:00.657855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 744/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:58:02.785398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:02.785973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:02.785973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:02.787271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 744/1488] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 10:58:02.792397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc68b60-f009-4a32-9bb4-07e5ef480e9a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:02.792878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc68b60-f009-4a32-9bb4-07e5ef480e9a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:02.792975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc68b60-f009-4a32-9bb4-07e5ef480e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:02.857892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7cc68b60-f009-4a32-9bb4-07e5ef480e9a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:58:02.859028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7cc68b60-f009-4a32-9bb4-07e5ef480e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:02.910492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:02.954451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 745/1489] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:58:02.994245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:58:02.996304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:03.031433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:03.097692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15982c54-be71-4bd1-a3d1-98afe550c94c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:58:03.167925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:58:03.168457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:03.168729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:03.168993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:03.170561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:03.248003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:03.277537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:58:03.278942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4ca73-35b7-428f-8397-be795cc4eaa2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:58:03.296605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 745/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:58:05.631400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:05.631868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:05.631992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:05.633352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 746/1491] 10.222.0.48 () {54 vars in 927 bytes} [Thu Aug 11 10:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:58:05.638515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafad4-cdb5-42a2-a1d1-cdea016ac77f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:05.638925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafad4-cdb5-42a2-a1d1-cdea016ac77f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:05.639037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafad4-cdb5-42a2-a1d1-cdea016ac77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:05.703067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-28aafad4-cdb5-42a2-a1d1-cdea016ac77f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:58:05.704271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28aafad4-cdb5-42a2-a1d1-cdea016ac77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:05.801131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 746/1492] 10.222.0.48 () {58 vars in 1024 bytes} [Thu Aug 11 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:58:05.840604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:58:05.842123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:05.944036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b4c8-7066-476b-8783-8ebf4096cb45 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:58:06.016724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:58:06.017398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:58:06.017675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:58:06.017942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:58:06.019664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:58:06.029333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:06.127946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:58:06.129347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225018-1d27-4d42-84ad-76ebdc5a0ac9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:58:06.147203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 747/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:58:15.799285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:58:16.145082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=50 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:00:08.773855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:08.774499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:08.774499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:08.855852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:00:08.856947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:08.957506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 747/1494] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:00:08.970370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:08.970888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:08.970999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:08.990957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:00:08.992170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:09.000547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 748/1495] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:00:09.005093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.014551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70381e9-c8da-4d68-aa5a-b3aa7f30c5d5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.019097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.019678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.020647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.020919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.021557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:00:09.021854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.022863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.022863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:00:09.022863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:00:09.023739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.035342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 748/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:00:09.044225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.053377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35ec8-6f1c-4bf6-a6ff-ab9566bfa6d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.058001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.058590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.058870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.059150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.059786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.061541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.061541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:00:09.061541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:00:09.062730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.193707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 749/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:00:09 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:00:09.198328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.208698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80b3b-c740-4f30-b71d-c85ceeb60775 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.257465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:00:09.264192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.264793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.265100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.265378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.265976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.266979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.266979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:00:09.266979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:00:09.268196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.374694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 749/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:00:09 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:00:09.378747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.389451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec28a4c-db87-41a8-bb7a-76f792f838cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.394039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.394658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.394933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.395897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.396532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.397523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.397523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:00:09.397523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:00:09.398396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.411015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 750/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:00:09.414938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.424656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939101-5ce8-42ee-b184-089e7ed83952 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.429832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.430417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.430697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.430969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.431603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.432635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.432635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:00:09.432635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:00:09.433514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aceda2-1bef-40d2-885f-34a623e5742c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.447774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 750/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:00:09.451830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.461753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9121-7bfd-4cf8-ab8a-9b744fca70fb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.466341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.466974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.468503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.468503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] PATH_INFO: `/v3/projects/e433894ddcc94f54b58c0a8dbfafd6a5/users/748b4b1d6c9b4503a225297e3c185cbb/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.482395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e433894ddcc94f54b58c0a8dbfafd6a5, user_id=748b4b1d6c9b4503a225297e3c185cbb, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.483420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.483420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:00:09.483420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:00:09.485388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0ebec-8a3c-4a4a-8aa7-527399748c91 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.556248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 751/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:00:09 2022] PUT /identity/v3/projects/e433894ddcc94f54b58c0a8dbfafd6a5/users/748b4b1d6c9b4503a225297e3c185cbb/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:00:09.568857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f94-01d0-48f5-9fc9-8e334b70c43a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.569146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f94-01d0-48f5-9fc9-8e334b70c43a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.569416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa1f94-01d0-48f5-9fc9-8e334b70c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.598174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4dfa1f94-01d0-48f5-9fc9-8e334b70c43a None None] MFA Rules not processed for user `748b4b1d6c9b4503a225297e3c185cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:00:09.599367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4dfa1f94-01d0-48f5-9fc9-8e334b70c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:09.660936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 751/1502] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:00:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:00:09.674011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0105b-843c-4a41-9f79-d29a26608f11 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.674297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0105b-843c-4a41-9f79-d29a26608f11 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.674570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0105b-843c-4a41-9f79-d29a26608f11 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.675949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 752/1503] 10.222.0.48 () {54 vars in 964 bytes} [Thu Aug 11 11:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:00:09.681177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79df7a-58f1-456d-8d84-5a60a52ad412 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.681467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79df7a-58f1-456d-8d84-5a60a52ad412 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.681738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79df7a-58f1-456d-8d84-5a60a52ad412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.737760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2b79df7a-58f1-456d-8d84-5a60a52ad412 None None] MFA Rules not processed for user `a1e9a4ceebcc44389144522cec3cc678`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:00:09.739374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b79df7a-58f1-456d-8d84-5a60a52ad412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:09.812389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 752/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Thu Aug 11 11:00:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:00:09.820563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0105b-843c-4a41-9f79-d29a26608f11 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.831239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0105b-843c-4a41-9f79-d29a26608f11 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.835233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.835884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.836162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.836432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.844370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.845772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:09.854294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 753/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:00:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:00:09.900106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89a77-dbdb-456f-baa4-8e57dcbf088d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.900397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89a77-dbdb-456f-baa4-8e57dcbf088d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.900676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89a77-dbdb-456f-baa4-8e57dcbf088d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.901938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 753/1506] 10.222.0.48 () {52 vars in 830 bytes} [Thu Aug 11 11:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:00:09.905465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:09.965613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656656-765d-411c-9b59-383acb583b48 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:09.970159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:09.970769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:09.971308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:09.971308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] PATH_INFO: `/v3/users/748b4b1d6c9b4503a225297e3c185cbb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:09.972424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=748b4b1d6c9b4503a225297e3c185cbb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:09.973446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:09.973446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:00:09.973446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:00:09.974630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:10.175677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748b4b1d6c9b4503a225297e3c185cbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:10.176855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748b4b1d6c9b4503a225297e3c185cbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:10.190630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748b4b1d6c9b4503a225297e3c185cbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:10.213520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748b4b1d6c9b4503a225297e3c185cbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:10.217434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:00:10.389601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748b4b1d6c9b4503a225297e3c185cbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:10.395736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 754/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:00:09 2022] DELETE /identity/v3/users/748b4b1d6c9b4503a225297e3c185cbb => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:00:10.400316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:10.400664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:10.400942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:10.419285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:00:10.420552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:10.430007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:00:10.476932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 754/1508] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:00:10.487168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:10.495204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:10.569871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e10225-28e9-45dc-bd44-9f05c39596fa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:10.573724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:10.574332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:10.574613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:10.574886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:10.583741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:10.585195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e0192-4ab3-4eb4-b405-612d27d6c8eb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:10.594497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 755/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:00:11.085703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:00:11.087214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:00:11.103841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842394c-dc77-42d0-a4d5-3c2ef7797a7a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:00:11.105080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:00:11.105671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:00:11.105950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:00:11.106220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] PATH_INFO: `/v3/projects/e433894ddcc94f54b58c0a8dbfafd6a5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:00:11.107350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e433894ddcc94f54b58c0a8dbfafd6a5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:00:11.108412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:00:11.108412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:00:11.108412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:00:11.109788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:00:11.215393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e433894ddcc94f54b58c0a8dbfafd6a5', 'request_id': 'req-0bfa689d-964f-4691-8730-1826163a51d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:11.231155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e433894ddcc94f54b58c0a8dbfafd6a5', 'request_id': 'req-0bfa689d-964f-4691-8730-1826163a51d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:11.340760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e433894ddcc94f54b58c0a8dbfafd6a5', 'request_id': 'req-0bfa689d-964f-4691-8730-1826163a51d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:11.352122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] The token cache is being invalidate because project e433894ddcc94f54b58c0a8dbfafd6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:00:11.352411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:11.353136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0bfa689d-964f-4691-8730-1826163a51d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:00:11.354635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 755/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:00:11 2022] DELETE /identity/v3/projects/e433894ddcc94f54b58c0a8dbfafd6a5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:00:20.590347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:00:21.349162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:01:42.478551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:42.479188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:42.479188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:42.480505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 11:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:01:42.485311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a914a-8d77-4b2b-8ee1-9b5256c7f645 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:42.485761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a914a-8d77-4b2b-8ee1-9b5256c7f645 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:42.485859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a914a-8d77-4b2b-8ee1-9b5256c7f645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:42.506172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-269a914a-8d77-4b2b-8ee1-9b5256c7f645 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:42.507387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-269a914a-8d77-4b2b-8ee1-9b5256c7f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:42.566494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:42.720326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:01:42.722039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:42.821104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d211637-6588-4bc4-9692-3fdd6ab5e3b9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:01:42.826375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:01:42.826963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:42.827253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:42.827506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:42.835275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:01:42.836863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6fc35-9990-4291-aa45-ac191656bdd4 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:01:42.845936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 757/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:01:43.680823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b3a88-00a8-4721-a032-87043850afe9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:43.681286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b3a88-00a8-4721-a032-87043850afe9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:43.681398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b3a88-00a8-4721-a032-87043850afe9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:43.682722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 11:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:01:43.687571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e45a5-c0d0-487b-bccf-cbd7d1abb8cd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:43.688042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e45a5-c0d0-487b-bccf-cbd7d1abb8cd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:43.688146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e45a5-c0d0-487b-bccf-cbd7d1abb8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:43.707793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-555e45a5-c0d0-487b-bccf-cbd7d1abb8cd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:43.708971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-555e45a5-c0d0-487b-bccf-cbd7d1abb8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:43.716607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 11:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:43.871593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b3a88-00a8-4721-a032-87043850afe9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:01:43.887042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b3a88-00a8-4721-a032-87043850afe9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:01:43.892393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:01:43.892989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:43.893264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:43.893535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:43.902015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:01:43.903603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db16a7c-4be4-4522-8774-fbe9834d0c87 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:01:43.913392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 758/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:01:44.329462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7f66a-aea0-4cb4-81f1-807a4603b84e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:44.330266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7f66a-aea0-4cb4-81f1-807a4603b84e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:44.330541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7f66a-aea0-4cb4-81f1-807a4603b84e None None] PATH_INFO: `/v3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:44.331891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 759/1517] 10.222.0.44 () {54 vars in 925 bytes} [Thu Aug 11 11:01:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:01:44.339251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2afb-e6e9-46a9-b289-161583b10086 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:44.339585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2afb-e6e9-46a9-b289-161583b10086 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:44.339814 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda2afb-e6e9-46a9-b289-161583b10086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:44.389730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-efda2afb-e6e9-46a9-b289-161583b10086 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:44.390925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-efda2afb-e6e9-46a9-b289-161583b10086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:44.460402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 759/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Aug 11 11:01:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:53.702082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030da4-d149-4af7-9ba2-f7732497d751 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:53.702612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030da4-d149-4af7-9ba2-f7732497d751 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:53.702808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=49 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:01:53.703353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48030da4-d149-4af7-9ba2-f7732497d751 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:53.704873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 760/1519] 10.222.0.48 () {54 vars in 961 bytes} [Thu Aug 11 11:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:01:53.710713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9866-6dc5-4119-90a7-3ced04e87d19 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:53.711147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9866-6dc5-4119-90a7-3ced04e87d19 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:53.711252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c9866-6dc5-4119-90a7-3ced04e87d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:53.730117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-827c9866-6dc5-4119-90a7-3ced04e87d19 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:53.731345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-827c9866-6dc5-4119-90a7-3ced04e87d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:53.739916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 760/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Aug 11 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:53.747250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48030da4-d149-4af7-9ba2-f7732497d751 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:01:53.760678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48030da4-d149-4af7-9ba2-f7732497d751 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:01:53.765578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:01:53.766162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:53.766439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:53.766708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:53.774816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:01:53.776409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365598e-cc22-4ba2-9676-19f1190785a6 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:01:53.786246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 761/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:01:55.152024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578dec0-bc38-4377-bf5c-c719cc73325b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:55.152623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578dec0-bc38-4377-bf5c-c719cc73325b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:55.152623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578dec0-bc38-4377-bf5c-c719cc73325b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:55.154100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 11:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:01:55.159366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf11e-c02a-4279-bca1-43b663550190 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:55.159848 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf11e-c02a-4279-bca1-43b663550190 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:55.159949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf11e-c02a-4279-bca1-43b663550190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:55.179989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1bcdf11e-c02a-4279-bca1-43b663550190 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:55.181182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1bcdf11e-c02a-4279-bca1-43b663550190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:55.189166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:55.344919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578dec0-bc38-4377-bf5c-c719cc73325b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:01:55.358876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578dec0-bc38-4377-bf5c-c719cc73325b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:01:55.364686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:01:55.365268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:55.365541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:55.365832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:55.373810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:01:55.375357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba98ee-9373-4673-9926-0c0697fc44f6 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:01:55.385152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 762/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:01:56.202993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf43e8-bb8e-4cb1-9c7d-3c3a1e59a551 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:56.203568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf43e8-bb8e-4cb1-9c7d-3c3a1e59a551 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:56.203568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf43e8-bb8e-4cb1-9c7d-3c3a1e59a551 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:56.204914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 11:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:01:56.210254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296cc9e-7f30-4943-8a95-042edf4ea3ca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:56.210671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296cc9e-7f30-4943-8a95-042edf4ea3ca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:56.210841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296cc9e-7f30-4943-8a95-042edf4ea3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:56.230194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b296cc9e-7f30-4943-8a95-042edf4ea3ca None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:01:56.231378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b296cc9e-7f30-4943-8a95-042edf4ea3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:01:56.239719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:01:56.390105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf43e8-bb8e-4cb1-9c7d-3c3a1e59a551 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:01:56.404502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf43e8-bb8e-4cb1-9c7d-3c3a1e59a551 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:01:56.409887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:01:56.410481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:01:56.410760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:01:56.411031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:01:56.419058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:01:56.420683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946984c3-9ab8-4921-a810-9ac249677f06 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:01:56.430432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 764/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:02:00.418890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:00.419436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:00.419436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:00.420765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:02:00.425974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d863-f29c-4b5c-9713-c1e73e5896ea None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:00.426424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d863-f29c-4b5c-9713-c1e73e5896ea None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:00.426521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d863-f29c-4b5c-9713-c1e73e5896ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:00.448343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1f28d863-f29c-4b5c-9713-c1e73e5896ea None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:02:00.449530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f28d863-f29c-4b5c-9713-c1e73e5896ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:00.457769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:02:00.532399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:02:00.534120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:00.561408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:02:00.591056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a2646-083c-49c7-8eb2-2e3a35026507 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:02:00.594748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:02:00.595572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:00.596175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:00.596456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:00.605613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:02:00.607047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31566ea2-d789-450f-965f-8313e5add22b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:02:00.616877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 765/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:02:02.608618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6a80-93a2-49db-ba49-7cc2cf817575 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:02.609049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6a80-93a2-49db-ba49-7cc2cf817575 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:02.609154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6a80-93a2-49db-ba49-7cc2cf817575 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:02.610518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 11:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:02:02.615732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de863-bc92-4fd3-8ae3-9ae2922e4c42 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:02.616195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de863-bc92-4fd3-8ae3-9ae2922e4c42 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:02.616309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56de863-bc92-4fd3-8ae3-9ae2922e4c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:02.635843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f56de863-bc92-4fd3-8ae3-9ae2922e4c42 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:02:02.637040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f56de863-bc92-4fd3-8ae3-9ae2922e4c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:02.645397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:02:02.680373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c6a80-93a2-49db-ba49-7cc2cf817575 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:02:02.691753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c6a80-93a2-49db-ba49-7cc2cf817575 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:02:02.695372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:02:02.695991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:02.696270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:02.696540 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:02.705899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:02:02.707341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624916e-ede1-43c7-8987-6c33e5b4c90e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:02:02.716574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 767/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:02:06.256542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:06.257118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:06.257118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:06.258403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 11:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:02:06.263650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296ab0-db08-4562-8bc8-1ac1e435c6a8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:06.264101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296ab0-db08-4562-8bc8-1ac1e435c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:06.264227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37296ab0-db08-4562-8bc8-1ac1e435c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:06.283550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-37296ab0-db08-4562-8bc8-1ac1e435c6a8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:02:06.284755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37296ab0-db08-4562-8bc8-1ac1e435c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:06.292395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 11:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:02:06.596795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:02:06.598498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:06.655717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a61cb3-2c08-443e-bc43-1c7a448226ba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:02:06.659307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:02:06.659944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:06.660226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:06.660495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:06.669276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:02:06.670684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9faf2-c5ff-4177-897f-3e5924675022 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:02:06.680354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 768/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:02:08.679601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:08.680199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:08.680199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:08.681514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 11:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:02:08.686703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f861b-f4d3-4744-babb-b77d2ad863e2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:08.687158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f861b-f4d3-4744-babb-b77d2ad863e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:08.687298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f861b-f4d3-4744-babb-b77d2ad863e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:08.707661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-943f861b-f4d3-4744-babb-b77d2ad863e2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:02:08.708911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-943f861b-f4d3-4744-babb-b77d2ad863e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:08.716741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:02:09.020979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:02:09.022613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:02:09.035892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd3c2-e01e-4e62-919d-e6c0554a7be7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:02:09.039288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:02:09.039902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:02:09.040178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:02:09.040444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:02:09.049286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:02:09.050683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e153181-0a9e-4666-abc8-702cfd9fc6b9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:02:09.059734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 770/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:02:18.701751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=47 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:02:19.055669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=33 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:33.290899 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6b28-0763-4e9b-bbc6-d45fd9090f77 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.293548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6b28-0763-4e9b-bbc6-d45fd9090f77 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.293548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6b28-0763-4e9b-bbc6-d45fd9090f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.317514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6bdb6b28-0763-4e9b-bbc6-d45fd9090f77 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.318733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6bdb6b28-0763-4e9b-bbc6-d45fd9090f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.330823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 770/1540] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.349808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.350450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.350450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.375456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f98-f282-40b9-9279-bcdd09fcfdd9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.375855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f98-f282-40b9-9279-bcdd09fcfdd9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.376047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344f98-f282-40b9-9279-bcdd09fcfdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.383866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.385598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.400062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 771/1541] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.405216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.406912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b344f98-f282-40b9-9279-bcdd09fcfdd9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.408329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b344f98-f282-40b9-9279-bcdd09fcfdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.417541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 771/1542] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.420522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320b67-74c9-44a2-a8fa-c2941536903a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.433898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8e691-c32f-47b1-82c7-4c8daf40149a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.434251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8e691-c32f-47b1-82c7-4c8daf40149a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.434455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8e691-c32f-47b1-82c7-4c8daf40149a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.435576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.437535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.437819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.438092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.438761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:33.439067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.440165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:33.440165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:03:33.440165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:03:33.444971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec4455-c629-4345-bc1e-a682e435be73 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.462715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2fa8e691-c32f-47b1-82c7-4c8daf40149a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.463181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 772/1543] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:03:33.463990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2fa8e691-c32f-47b1-82c7-4c8daf40149a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.464643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.465010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.465295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.478489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 772/1544] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:03:33.479802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.480094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.480368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.506686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.508102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.518272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.519532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.524186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 773/1545] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.525043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.537891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 773/1546] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.539484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.551260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65923e43-c5fd-4cfb-8e0a-9e4c935c0199 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.560550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c47df-1873-4a4c-95c6-34029a8db15e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.563266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.563925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.564979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.565263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.566350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.570360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.574408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.575028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.575307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.575579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.576329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:33.576636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.580847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3640c-7995-469b-b6b5-49f46a2ea852 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.595408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 774/1547] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:33.597138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5672634-1772-459c-a488-cc353cc1aad1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.597607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5672634-1772-459c-a488-cc353cc1aad1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.597886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5672634-1772-459c-a488-cc353cc1aad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.623456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a5672634-1772-459c-a488-cc353cc1aad1 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.628107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5672634-1772-459c-a488-cc353cc1aad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.635564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 775/1548] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.637244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734236-fe09-48c6-bb18-f0e866c53007 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.637710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734236-fe09-48c6-bb18-f0e866c53007 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.638283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734236-fe09-48c6-bb18-f0e866c53007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.663333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87734236-fe09-48c6-bb18-f0e866c53007 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.664581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87734236-fe09-48c6-bb18-f0e866c53007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.673885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 776/1549] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.675835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b356d-b46e-4822-a643-adc93586cd73 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.675835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b356d-b46e-4822-a643-adc93586cd73 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.675835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b356d-b46e-4822-a643-adc93586cd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.688116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 774/1550] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:33.688815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.704757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-134b356d-b46e-4822-a643-adc93586cd73 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.705968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-134b356d-b46e-4822-a643-adc93586cd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.763038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668168d5-d8fd-4e79-ae46-87ef934ca6f6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.766878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 777/1551] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.770594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.770726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076588d-ab57-4735-918d-9185a27a77a1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.770726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076588d-ab57-4735-918d-9185a27a77a1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.770884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.770968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076588d-ab57-4735-918d-9185a27a77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.771087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.771087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.771087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.771834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.787366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e076588d-ab57-4735-918d-9185a27a77a1 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.788642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e076588d-ab57-4735-918d-9185a27a77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.797128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 778/1552] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.798531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.798531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.798809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.818117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.819206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.827162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 779/1553] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.828087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.837644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a90ef-6d5f-45a8-9958-b8e74eef40e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.843575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.845460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.845460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.845460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.845460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:33.845824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.847147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81d9cb-40e4-43bf-8a30-43e7cc0e5f6b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.859436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 780/1554] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:33.860674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d378-b8e8-40c7-835b-f2e7aaeafd6d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.860955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d378-b8e8-40c7-835b-f2e7aaeafd6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.861320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d378-b8e8-40c7-835b-f2e7aaeafd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.872062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 775/1555] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:33.874138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.888817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-79d4d378-b8e8-40c7-835b-f2e7aaeafd6d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.890113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79d4d378-b8e8-40c7-835b-f2e7aaeafd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.940746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb2575-5dc7-41f5-b54c-3dddab0e5221 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.945568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.946171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.946456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.946729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.947346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.948392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:33.948392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:03:33.948392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:03:33.949594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a899dd-fff8-4419-b525-ac7dac3c5bef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:33.950018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 781/1556] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.951245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.951524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.951815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.972340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:33.973514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:33.981284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 782/1557] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:33.982031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:33.991374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfd2b2-9e23-4170-a04f-59f737b79519 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:33.995937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:33.996502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:33.996775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:33.997041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:33.997660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:33.997958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:33.999514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.010672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 783/1558] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:34.011425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.015954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:34.020812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a5be3-c3b4-417a-8b77-711b8ec9e4e6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.025294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.027108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.027108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.027108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.027108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:34.027496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.028685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 776/1559] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:33 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.028928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.029942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.031088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.031370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.048744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 784/1560] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:34.048744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.053546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a37a-881b-4062-93e9-7d934f142759 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.054640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:34.055921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:34.058236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.058815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.059088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.059360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.059994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.060994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:34.060994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:03:34.060994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:03:34.062500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2d144-a524-471d-ab7d-6ea709d43e3a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.064336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 777/1561] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:34.065160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.074774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3f4cf-7850-4ca2-a3f6-570876aea062 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.079462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.080112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.080391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.080660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.081293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.083156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711a784-3956-461a-ab56-e43c917fb3ce admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.137852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 785/1562] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.139280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.139569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.139896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.159904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:34.161085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:34.178931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 778/1563] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.178931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.178931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.178931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.184749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:34.196453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:34.197637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:34.222415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 786/1564] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:34.223280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.232167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d759f38-02bf-4c4f-9bb4-e4e1bb78c4eb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.236656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.237238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.237511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.237779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.238691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.240302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.256283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 779/1565] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:34.257274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.267307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f120f-a7b4-4a2a-9d54-ffcffd5275fc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.271732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.272374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.272645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.272912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.273566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.275123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.284719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 780/1566] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:34.285508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.294428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6b2c2-2d32-4f97-b84d-739bd3615cd7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.299310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 787/1567] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.299310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.299980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.299980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.305756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.305756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.305756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.305756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.361486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118d937-ffc9-4a34-9659-e1db64273dea admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.365985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.366916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.366916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.367219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.367748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:34.368071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.369694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.381779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 788/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:34.382612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.392356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2dc22-0074-4c9c-be77-d6312423abb8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.395009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 781/1569] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.395871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.397605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.398291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.398565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.398834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.399444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.401376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.455748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90195c7-409b-4e72-9234-f2f965725484 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.460248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.460861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.461140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.461409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.462030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.463593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.473118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 782/1570] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:34.476416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.482684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906af85-ca84-4c36-a61b-f7195b7fda45 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.487875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.488405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.488679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.488961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.489538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:34.489832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.491288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.501757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 789/1571] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.502692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.511380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 783/1572] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:34.511380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.512910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895011d5-d4c9-4296-8f2b-2aa0c0074b4a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.517574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.522214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.522798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e22730-30b4-4fa9-93ad-69dd4e203f02 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.526212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.526800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.527083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.527354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.528013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.529570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.533882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 790/1573] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:34.534725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.539820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 784/1574] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:34.540816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.546787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62886db4-87e8-48f2-9edc-af444d097f0c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.550924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cbd3b-641e-47b5-bcf1-3d93cad95f1f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.551227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.551838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.552116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.552383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.552986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.554869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.555570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.556194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.556472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.556743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.557351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.559230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.564577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:34.677628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 791/1575] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/users => generated 282 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.677628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.677908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 785/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.677908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.688040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033812e7-9598-4746-9c97-dad44d18f1e4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.697597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:34.738468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a560eb7-0053-49ec-8ce3-9ea7f95cc687 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.743593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.744262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.744425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.744545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.744792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.744992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.745272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.745427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.745549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.746268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.746997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.748206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.756969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 786/1577] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:34.757835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.767079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ed05d-382d-4215-a283-7d1f91a77171 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.772164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.774438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.774438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.774438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.774438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.775413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.785747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 787/1578] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:34.786677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.795688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5c6ed-8c4e-4079-a267-98c032843e27 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.800787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.801327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.801603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.801871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.802457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.804336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.822452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 792/1579] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.823263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.832887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e2f95-7879-4ae5-bef1-af0c1297d6e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.837470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.838062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.838328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.838603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.839209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.841145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:34.886767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 788/1580] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.887724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.930377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 793/1581] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:34 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:34.930377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:34.951350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ac13b-99a9-43d9-a5ba-b1bf350ded88 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.988760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851a503-8fb3-4313-b895-f47980977ea0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:34.993372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:34.993962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:34.994236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:34.994511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:34.995385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:34.996743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.006324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 794/1582] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.006455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.007048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.007327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.007600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] PATH_INFO: `/v3/projects/8b25b6f3331a4ad49c7312a3a4bc3154/users/7571d52d0a1f4aa4a9d0dd6dfe2c7a76/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.008642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.018863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8674133-9dbb-485d-b719-dd74d119eb83 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.022334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b25b6f3331a4ad49c7312a3a4bc3154, user_id=7571d52d0a1f4aa4a9d0dd6dfe2c7a76, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.023484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.024099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.024384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.024660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.024997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.025263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.027141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.090890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 789/1583] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:34 2022] PUT /identity/v3/projects/8b25b6f3331a4ad49c7312a3a4bc3154/users/7571d52d0a1f4aa4a9d0dd6dfe2c7a76/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:35.091686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.102071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 795/1584] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:35.103418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.109039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:35.113945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1327160-89f2-4d64-afac-a6b994156a43 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.148009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:35.157179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d03fbe-f7e3-4b1b-9f63-859b40314643 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.161785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.162526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.163356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.163989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] PATH_INFO: `/v3/projects/74eff4ac3477460cb61a800a3a1add39/users/9350407241ea496fa23cf9a883cb2b04/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.171071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.174063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.174063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.174063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.174063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.174510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.178586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74eff4ac3477460cb61a800a3a1add39, user_id=9350407241ea496fa23cf9a883cb2b04, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.181290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.184141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 796/1585] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.185227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.196578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74abbdd-6eaf-4815-8b22-47616140575b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.201353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.201965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.202750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.203026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.203678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.205331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.214852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 797/1586] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.215668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.225256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddff0e-51c1-4e72-bceb-36b967bdf1bb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.229675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.230228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.230509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.230782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.231370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.233236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.273777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 790/1587] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:35 2022] PUT /identity/v3/projects/74eff4ac3477460cb61a800a3a1add39/users/9350407241ea496fa23cf9a883cb2b04/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:35.274582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.333383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 798/1588] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:35.334375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.343459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7615b08-9465-4390-a02f-d7280f5b7e39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.344540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795ec33-97cb-473d-bca0-3a6733f19f27 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.348176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.348770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.348951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.349078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.349302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.349544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.349813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.349923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.350084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.350712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.351811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.352411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52a1a4-ec0d-4be6-8304-1ea2466118f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.363109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 799/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.364432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.364717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.364987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.391635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] MFA Rules not processed for user `7571d52d0a1f4aa4a9d0dd6dfe2c7a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:35.392884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:35.450765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 791/1590] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:35.451674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.460279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 800/1591] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:35.460279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.470756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda221-af41-4860-b129-acc454e262c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.470756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.470756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.470756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.471126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.471126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.471126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.472285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a633ea-e42d-428b-b22f-1266451577ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.477729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.478375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.478681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.478917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] PATH_INFO: `/v3/projects/b0b3799012d24fad98d155e3b000fc22/users/bb18a00d44ed4af186bdc589e1e14e22/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.483906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 792/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.484945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.494779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514a3f2-98e7-4b5e-8cc6-dbe58e656c88 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.497052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b3799012d24fad98d155e3b000fc22, user_id=bb18a00d44ed4af186bdc589e1e14e22, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.498297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:35.498297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:03:35.498297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:03:35.500525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.501013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.501314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.501657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.502032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.502707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.504693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee04d95-ee4b-4839-a981-2aba91d3f636 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.515530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 793/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.517847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.517847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.517847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.542947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] MFA Rules not processed for user `9350407241ea496fa23cf9a883cb2b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:35.544172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:35.599456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:35.600033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 801/1594] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:35 2022] PUT /identity/v3/projects/b0b3799012d24fad98d155e3b000fc22/users/bb18a00d44ed4af186bdc589e1e14e22/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:35.604906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.615388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 794/1595] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:35.620356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.631865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557a69c-63a4-416a-818c-4b5bff93d678 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.669262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57494d20-e213-429a-a691-edeec7a7b055 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.676757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.677384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.677657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.677925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.678566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.680183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.681770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:35.691999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 802/1596] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.692524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.693280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.693813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.694103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] PATH_INFO: `/v3/projects/34c47c71c036488888c1bf5eba9d3855/users/2a086a2d99094464b316d6771ff3a1ba/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.696870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.712449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07371e-387a-498d-bb9a-05b7fd373c1a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.712779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c47c71c036488888c1bf5eba9d3855, user_id=2a086a2d99094464b316d6771ff3a1ba, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.712893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.712893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.712893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.713162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4d434-ee8d-4b5a-9890-89add6272d32 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.713259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.713259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.714338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.724182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 803/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.724182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.734477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfd9b4-9537-49c6-94d1-b3d0f9410363 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.738958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.739536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.739838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.740108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.740708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.742273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e4b18-1d30-43f5-a139-e8e09cf022da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.752683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 804/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:35.753835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad7ae-3ce4-47bd-b9f0-323482f4846d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.754114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad7ae-3ce4-47bd-b9f0-323482f4846d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.754380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad7ae-3ce4-47bd-b9f0-323482f4846d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.755556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 805/1599] 10.222.0.48 () {54 vars in 962 bytes} [Thu Aug 11 11:03:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:35.756252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad7ae-3ce4-47bd-b9f0-323482f4846d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.768122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad7ae-3ce4-47bd-b9f0-323482f4846d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.772545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.773147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.773422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.773694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] PATH_INFO: `/v3/projects/cbb076c769694966bbfea2f23ea8b8a4/users/2d6fc583bb8a4434b0c6538c9ffbde75/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.788754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 795/1600] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:35 2022] PUT /identity/v3/projects/34c47c71c036488888c1bf5eba9d3855/users/2a086a2d99094464b316d6771ff3a1ba/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:35.790254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.790736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.791190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.797290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb076c769694966bbfea2f23ea8b8a4, user_id=2d6fc583bb8a4434b0c6538c9ffbde75, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.800193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:35.815133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:35.826023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:35.875445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 796/1601] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:35.876834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.909363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 806/1602] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:35 2022] PUT /identity/v3/projects/cbb076c769694966bbfea2f23ea8b8a4/users/2d6fc583bb8a4434b0c6538c9ffbde75/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:35.910547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:35.945852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a01dd8-8b0d-44b9-b4c8-04f9f00fca3c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.979793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:35.981922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9392fd-6439-494d-bc9a-be9a8a149be4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:35.988400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:35.989175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:35.989481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:35.989759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:35.990407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:35.992105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.001199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.001963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.002241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.002699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.003260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 807/1603] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:36.004047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.017623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14212569-bab9-4c0d-b0ed-4b0e12b845e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.022697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.023441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.023737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.024079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.024671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.026271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.039929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 808/1604] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:36.040872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.051374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300b2c9-e67a-437b-93f0-a3cb4cb76383 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.056073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.056718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.056998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.057272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] PATH_INFO: `/v3/projects/d11d73b282114c02bed69938677e2e58/users/b226ddd101ae4ff9aeab0fc797a349f5/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.073575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.074701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11d73b282114c02bed69938677e2e58, user_id=b226ddd101ae4ff9aeab0fc797a349f5, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.075369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af353a-8f6d-422a-8bcf-f6d786fe7995 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.077491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032bc11-0e6c-44c0-a15e-de42025b197d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.085780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 797/1605] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:36.087119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.087481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.087775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.132774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.133975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.178264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 809/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:36 2022] PUT /identity/v3/projects/d11d73b282114c02bed69938677e2e58/users/b226ddd101ae4ff9aeab0fc797a349f5/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:36.180809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7563f6-912e-453c-8fa5-020f54e03d86 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.180809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7563f6-912e-453c-8fa5-020f54e03d86 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.180809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7563f6-912e-453c-8fa5-020f54e03d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.215157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3a7563f6-912e-453c-8fa5-020f54e03d86 None None] MFA Rules not processed for user `2a086a2d99094464b316d6771ff3a1ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.216498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a7563f6-912e-453c-8fa5-020f54e03d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.228247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 798/1607] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:36.229326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.289849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:36.290531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41fa0b-2beb-4431-bdef-b1218c38abc2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.294227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 810/1608] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:36.295663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.296781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.297347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.322988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.323707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:36.324732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.363057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.364215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.364472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.365043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.392011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 811/1609] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:36.393900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.402905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d42fd3-1908-490f-8d3e-0e403f3421ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.408899 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.409720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.410174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.410631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] PATH_INFO: `/v3/projects/c42a3b4ef2d644b58856779226688f88/users/1b7724f3ef4543ed9c3df0455ae4fb76/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.427804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42a3b4ef2d644b58856779226688f88, user_id=1b7724f3ef4543ed9c3df0455ae4fb76, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.432652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30290125-ef8b-407b-b2c1-27aa78d11950 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.444762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.447735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.458610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 799/1610] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:36.459391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.470395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40792d82-a1f7-4496-85de-c55c1fc195ae service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.475211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.475898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.476268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.476544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] PATH_INFO: `/v3/projects/e8751d4bd74240db9b57b7e9f788df94/users/684d59041e82466d8aad6d2964842bf5/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.491734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8751d4bd74240db9b57b7e9f788df94, user_id=684d59041e82466d8aad6d2964842bf5, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.494395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.524983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 812/1611] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:36 2022] PUT /identity/v3/projects/c42a3b4ef2d644b58856779226688f88/users/1b7724f3ef4543ed9c3df0455ae4fb76/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:36.526142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.526427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.526699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.559523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] MFA Rules not processed for user `b226ddd101ae4ff9aeab0fc797a349f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.560866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.611558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 800/1612] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:36 2022] PUT /identity/v3/projects/e8751d4bd74240db9b57b7e9f788df94/users/684d59041e82466d8aad6d2964842bf5/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:36.612976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.651640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 813/1613] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:36.652525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.702948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bf5d0-160d-497c-8173-46989dce211a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.709119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.711505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.711505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.711505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.725805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4012c5-eb89-4cff-adf0-9e26794a05aa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.736112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:36.758310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:36.770593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.781003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.781930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 801/1614] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:36.782628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:36.783942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.790722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.791824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.792120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.792394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.793314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.796459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553452-386d-4154-ae70-1526fc761433 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.847481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a72b3-a3cd-4e81-a0c6-f71f0b739158 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:36.852589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:36.853256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.853530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.853799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.863487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:36.866239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e0a87-5dae-44cc-9e71-c44703132e8e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:36.888305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 802/1615] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:36.889887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8066-b06d-4138-8e65-a87da576d748 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.890326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8066-b06d-4138-8e65-a87da576d748 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.890748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8066-b06d-4138-8e65-a87da576d748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.911310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c3c8066-b06d-4138-8e65-a87da576d748 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.912584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c3c8066-b06d-4138-8e65-a87da576d748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.913071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 814/1616] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/projects => generated 363 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:36.914377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.914657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.914935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.945116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:36.946504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:36.983203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 803/1617] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:36.985691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cf1a-c880-4946-ac62-fb1077e05d50 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:36.994948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cf1a-c880-4946-ac62-fb1077e05d50 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:36.995381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7cf1a-c880-4946-ac62-fb1077e05d50 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:36.997187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 804/1618] 10.222.0.48 () {54 vars in 962 bytes} [Thu Aug 11 11:03:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:36.998400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7cf1a-c880-4946-ac62-fb1077e05d50 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.031128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7cf1a-c880-4946-ac62-fb1077e05d50 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.037254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.037952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.038501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.038860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.039556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.041660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19091ce8-3ebc-48b8-bc1c-d669da500ad5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.070012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 815/1619] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:37.071131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.135718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 805/1620] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:36 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:37.137621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.137621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.137621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.168355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:37.172799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:37.175415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0654b-9ea0-4945-be69-4cd37543cc56 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.178672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.179362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.179714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.180121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.193497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.196080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.235700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 816/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:37.238247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.258050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e43f-1187-4773-a96c-0f0810ce61d5 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.266115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.267389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.270157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.270157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.301281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.305230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af51f31-18b8-4ddd-95bb-f7628845a985 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.314148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 806/1622] 10.222.0.48 () {58 vars in 1059 bytes} [Thu Aug 11 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:37.315046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.318580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 817/1623] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:37.320200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c242e-ee34-43ac-9b29-1c76ec799b86 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.321078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c242e-ee34-43ac-9b29-1c76ec799b86 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.321360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c242e-ee34-43ac-9b29-1c76ec799b86 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.322788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 818/1624] 10.222.0.48 () {54 vars in 912 bytes} [Thu Aug 11 11:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:37.324293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c242e-ee34-43ac-9b29-1c76ec799b86 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.342045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c242e-ee34-43ac-9b29-1c76ec799b86 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.350051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.350673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.350953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.351232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.352945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca39ef-b9ec-45cf-a7bb-4a36bc8cefd7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.365203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.365823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.366101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.366371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.367011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.368765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5ac7-694b-4113-911a-41a3b48b21b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.391593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 807/1625] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:37.395945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.396237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.396512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.422622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.425614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.441304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:37.441405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:37.441405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:37.444023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 819/1626] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:03:37.445103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.452302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 808/1627] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:37.457498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4f8-8aa4-4337-a3df-dce9337fdf3f service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.464660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.470844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.470844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.470844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] PATH_INFO: `/v3/projects/3d93ea156d4240619162154e3864e012/users/689d494727f54427aa20285445c3e647/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.471154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.486471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d93ea156d4240619162154e3864e012, user_id=689d494727f54427aa20285445c3e647, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.490203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e85400-18bc-45be-b992-581bcd42da7a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.551507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1f242-3e83-4891-87ba-d4bd269ca23f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.555018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.555619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.555934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.556205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.566204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:37.569125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.573935 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.583147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 809/1628] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:03:37.602610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 820/1629] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:37 2022] PUT /identity/v3/projects/3d93ea156d4240619162154e3864e012/users/689d494727f54427aa20285445c3e647/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:37.604024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.616822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554c17d-3c8d-4a59-a3c1-7058b43c0231 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad33bd-51aa-4175-843b-7af48a9006d0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad33bd-51aa-4175-843b-7af48a9006d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad33bd-51aa-4175-843b-7af48a9006d0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 821/1630] 10.222.0.48 () {54 vars in 912 bytes} [Thu Aug 11 11:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635572-c158-4877-88ff-320021af5694 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.733907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635572-c158-4877-88ff-320021af5694 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.734410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635572-c158-4877-88ff-320021af5694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.751124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.751753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.752056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.752333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.753177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.755922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.764340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-15635572-c158-4877-88ff-320021af5694 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:37.784771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15635572-c158-4877-88ff-320021af5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:37.787106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 810/1631] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:37.787106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.808285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a377-a1ce-4ccf-bfc3-92d1824441f5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.809749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.810357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.810639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.811043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] PATH_INFO: `/v3/projects/3d93ea156d4240619162154e3864e012/users/689d494727f54427aa20285445c3e647/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.813344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d93ea156d4240619162154e3864e012, user_id=689d494727f54427aa20285445c3e647, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.816447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.829217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 811/1632] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:37 2022] PUT /identity/v3/projects/3d93ea156d4240619162154e3864e012/users/689d494727f54427aa20285445c3e647/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:37.830067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:37.923267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 822/1633] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:37.925440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.926220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.926703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.938227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbeec1-ba11-4a77-aef5-9eefd89f0b68 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:37.943306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:37.944167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:37.945818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:37.945818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] PATH_INFO: `/v3/users/7571d52d0a1f4aa4a9d0dd6dfe2c7a76` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:37.945818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7571d52d0a1f4aa4a9d0dd6dfe2c7a76)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:37.947636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:37.960475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] MFA Rules not processed for user `689d494727f54427aa20285445c3e647`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:37.961875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:38.029923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 823/1634] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:38.032883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:38.091245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740f1a-291a-40a5-b5a3-fe78490039e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:38.100531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:38.100531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:38.100531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:38.100531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:38.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:38.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:38.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 824/1635] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:38.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:38.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63b315-30fc-4dc7-a477-c93e2270b09c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:38.387087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:38.387087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:38.387087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:38.387087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:38.391458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:38.391458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:38.391458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5258431-0add-4950-8a83-3ef515c629a6 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:38.392498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7571d52d0a1f4aa4a9d0dd6dfe2c7a76'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:38.392498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7571d52d0a1f4aa4a9d0dd6dfe2c7a76'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:38.392498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7571d52d0a1f4aa4a9d0dd6dfe2c7a76'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:38.393264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 825/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:38.393820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7571d52d0a1f4aa4a9d0dd6dfe2c7a76'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:38.393820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:38.456672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7571d52d0a1f4aa4a9d0dd6dfe2c7a76'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:38.464177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 812/1637] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:37 2022] DELETE /identity/v3/users/7571d52d0a1f4aa4a9d0dd6dfe2c7a76 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:38.475003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:38.475457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:38.475551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:38.508792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:38.511460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:38.611607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:38.613696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:38.622587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 826/1638] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:38.634770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:38.636720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:38.704716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17744278-d230-495a-9820-aac416450fd0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:38.732021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:38.733045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:38.734637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:38.736414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:38.742786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:38.758563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42007224-12a9-4905-966b-d001feeaefc2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:38.762586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:38.763316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:38.763715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:38.764111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:38.774022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:38.776053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:38.790634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 827/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:03:38.832030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:38.833962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a962d2-3174-479b-8075-9e8c1c6251d1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:38.845319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 813/1640] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:39.595065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:39.596877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:39.621194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09535-c1ca-429d-8ee1-025fb1659991 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:39.652547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:39.653391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:39.655730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:39.662156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] PATH_INFO: `/v3/projects/8b25b6f3331a4ad49c7312a3a4bc3154` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:39.675215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b25b6f3331a4ad49c7312a3a4bc3154)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:39.678931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:39.881049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b25b6f3331a4ad49c7312a3a4bc3154', 'request_id': 'req-66f34daa-2ac5-40e0-8993-9b2f534d419b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:39.883930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:39.894031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b25b6f3331a4ad49c7312a3a4bc3154', 'request_id': 'req-66f34daa-2ac5-40e0-8993-9b2f534d419b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:40.022658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b25b6f3331a4ad49c7312a3a4bc3154', 'request_id': 'req-66f34daa-2ac5-40e0-8993-9b2f534d419b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:40.038415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] The token cache is being invalidate because project 8b25b6f3331a4ad49c7312a3a4bc3154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:40.038986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:40.039856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-66f34daa-2ac5-40e0-8993-9b2f534d419b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:40.041528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 828/1641] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:39 2022] DELETE /identity/v3/projects/8b25b6f3331a4ad49c7312a3a4bc3154 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:40.047730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.048233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.048415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.072398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:40.073677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:40.246158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 814/1642] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:40.254087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.254786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.255282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.296292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:40.297543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:40.307207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 829/1643] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:40.312545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:40.326351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf6324-b4f3-4337-8366-43761c54be21 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:40.338293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:40.339114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.341208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.341208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.341208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:40.341208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:40.344540 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:40.363149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 815/1644] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:40.367597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:40.378811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be54a9-b158-4cc7-9855-e795b7417a99 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:40.393276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:40.393276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.393276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.393276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.395546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:40.397644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:40.541348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 830/1645] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:40 2022] POST /identity/v3/projects => generated 405 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:03:40.546976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:40.579985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f8bc-fbfd-4317-8e1b-d48cbfd752c8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:40.762108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:40.762927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.763217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.763490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.764246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:40.766393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:40.793956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:40.876720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 816/1646] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:40 2022] POST /identity/v3/users => generated 303 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:40.881297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:40.903723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcea40b-8103-4168-8bcb-49c08d7071a2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:40.927557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:40.928197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:40.928790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:40.929194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:40.930156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:40.931932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:40.969895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 831/1647] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:40.970970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:40.984397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55f84-0ede-488f-8d43-fec8a1d8d9f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:40.998221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:41.002187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:41.002187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:41.002187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:41.002187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:41.002187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553617fe-edc0-49dc-a4c9-e3340c0b0a2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:41.054015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 817/1648] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:41.055276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:41.082274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71fcda-80ff-4aab-a5e0-1dc22627f59e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:41.091323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:41.093141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:41.093429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:41.093748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] PATH_INFO: `/v3/projects/c9089fab4b8f48c4ab191703256e3e94/users/86f7f43c14004bc398df30f35aa7c8f4/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:41.131466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9089fab4b8f48c4ab191703256e3e94, user_id=86f7f43c14004bc398df30f35aa7c8f4, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:41.134901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:41.216834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 832/1649] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:41 2022] PUT /identity/v3/projects/c9089fab4b8f48c4ab191703256e3e94/users/86f7f43c14004bc398df30f35aa7c8f4/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:41.228593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:41.229156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:41.229602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:41.288945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] MFA Rules not processed for user `86f7f43c14004bc398df30f35aa7c8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:41.290691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:41.477958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 818/1650] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:41.482516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:41.640847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a38b3-8014-4e31-b78e-50b68442f5df admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:41.660192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:41.660996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:41.661358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:41.661789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:41.662494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:41.669775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:41.722103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:41.844355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 833/1651] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:41 2022] POST /identity/v3/projects => generated 403 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:41.848405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:41.862381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c2e9-f74c-437f-9680-9e2147ac468d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:42.023401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:42.025247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.026223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.026538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.027195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:42.030715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:42.147070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 819/1652] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:41 2022] POST /identity/v3/users => generated 308 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:42.152043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:42.169258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb053-3e4e-4aef-8455-f1a714080073 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:42.180323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:42.183240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.183529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.183886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.192205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:42.193860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:42.229316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 834/1653] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:42.230235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:42.245276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68632c-c00b-4bd5-bf11-bda51e9eb33d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:42.266621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:42.267343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.267610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.267939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] PATH_INFO: `/v3/projects/704c7146830a4c58930737948274860e/users/d9b3374628dc4ff4b928c77d022f41f9/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.324764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704c7146830a4c58930737948274860e, user_id=d9b3374628dc4ff4b928c77d022f41f9, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:42.328191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:42.417642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 820/1654] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:42 2022] PUT /identity/v3/projects/704c7146830a4c58930737948274860e/users/d9b3374628dc4ff4b928c77d022f41f9/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:42.421685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:42.444609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a00822-7a1a-4d60-b640-2a3dec0c1186 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:42.570316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:42.570316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.570316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.570316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.570848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:42.574666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b13623-9810-4817-a73e-36417c306b15 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:42.621598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 835/1655] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:42.627174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:42.646267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1c1bd-15cc-4b8b-8518-49059dcc9f72 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:42.653958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:42.659135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.660279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.660687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] PATH_INFO: `/v3/projects/704c7146830a4c58930737948274860e/users/d9b3374628dc4ff4b928c77d022f41f9/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.685402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704c7146830a4c58930737948274860e, user_id=d9b3374628dc4ff4b928c77d022f41f9, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:42.696725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:42.740034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:42.750077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 821/1656] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:42 2022] PUT /identity/v3/projects/704c7146830a4c58930737948274860e/users/d9b3374628dc4ff4b928c77d022f41f9/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:42.761885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:42.762364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:42.762904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:42.827733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] MFA Rules not processed for user `d9b3374628dc4ff4b928c77d022f41f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:42.830923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:42.989830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 836/1657] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:43.002697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:43.004854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:43.129981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31fb1-622e-470d-9de0-92d97ab73986 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:43.140356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:43.140991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:43.142415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:43.142696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:43.169243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:43.171587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3074f-b7b2-4341-a634-54af1861b489 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:43.201992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 822/1658] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:45.412282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:45.578816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420b210-1208-46b3-81f7-6078cf4ccd95 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:45.589725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:45.589725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:45.589725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:45.589725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] PATH_INFO: `/v3/users/86f7f43c14004bc398df30f35aa7c8f4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:45.596685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f7f43c14004bc398df30f35aa7c8f4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:45.601802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:45.601802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 11 11:03:45.601802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: Aug 11 11:03:45.607451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:45.635837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:45.835587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:45.841893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:45.841893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:45.877648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] MFA Rules not processed for user `bb18a00d44ed4af186bdc589e1e14e22`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:45.878939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:45.986568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f7f43c14004bc398df30f35aa7c8f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:45.988790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f7f43c14004bc398df30f35aa7c8f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:45.999595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 823/1659] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:46.013856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:46.017961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:46.065419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f7f43c14004bc398df30f35aa7c8f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:46.097999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:46.161086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f7f43c14004bc398df30f35aa7c8f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:46.205546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffc073-e980-4637-ad09-1dd424107917 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:46.214221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f62e4c0-0da6-43aa-95d1-5ce2b02bd2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f7f43c14004bc398df30f35aa7c8f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:46.217677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:46.217677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:46.217677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:46.217677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:46.219815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:46.229148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 837/1660] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:45 2022] DELETE /identity/v3/users/86f7f43c14004bc398df30f35aa7c8f4 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:46.234357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:46.234357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:46.234654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:46.243257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:46.246468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:46.295411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 824/1661] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:46.298367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:46.299907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:46.510975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 838/1662] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:46.530281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:46.531881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:46.709823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:46.710350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:46.725121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0d49b-3683-4159-b91a-ed642390b7bc service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:46.730822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce7f2-855c-4788-a7de-8cb051915f97 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:46.741632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:46.742469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:46.745052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:46.745588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] PATH_INFO: `/v3/users/2a086a2d99094464b316d6771ff3a1ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:46.747177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a086a2d99094464b316d6771ff3a1ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:46.751836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:46.815565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:46.816410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:46.816860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:46.817264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:46.836387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:46.841642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244492e-c583-4bcf-8d7d-19e000d38560 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:46.874769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 825/1663] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:47.100110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a086a2d99094464b316d6771ff3a1ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:47.100842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a086a2d99094464b316d6771ff3a1ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:47.139185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a086a2d99094464b316d6771ff3a1ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:47.221745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a086a2d99094464b316d6771ff3a1ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:47.233281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a086a2d99094464b316d6771ff3a1ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:47.237223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:47.249384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 839/1664] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:46 2022] DELETE /identity/v3/users/2a086a2d99094464b316d6771ff3a1ba => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:47.253969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:47.255621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:47.256017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:47.300538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:47.308622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:47.547074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 826/1665] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:47.576456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:47.578055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:47.757186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f7389-6ec0-4bd8-9657-a82cb9dfad61 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:47.769769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:47.770419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:47.770695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:47.770962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:47.820437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:47.821907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:47.839164 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 840/1666] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:48.381415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:48.384820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:48.409952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2148d0-5ef6-471a-aa00-071626b716d5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:48.425069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:48.427476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:48.428040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:48.428519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] PATH_INFO: `/v3/projects/c9089fab4b8f48c4ab191703256e3e94` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:48.430362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9089fab4b8f48c4ab191703256e3e94)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:48.431585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 11:03:48.431585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 11 11:03:48.431585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: Aug 11 11:03:48.436442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:48.440637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:48.738442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9089fab4b8f48c4ab191703256e3e94', 'request_id': 'req-d9c17201-808b-49de-a9b6-0165b99aa5bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:48.773341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9089fab4b8f48c4ab191703256e3e94', 'request_id': 'req-d9c17201-808b-49de-a9b6-0165b99aa5bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:48.890276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9089fab4b8f48c4ab191703256e3e94', 'request_id': 'req-d9c17201-808b-49de-a9b6-0165b99aa5bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:48.915565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] The token cache is being invalidate because project c9089fab4b8f48c4ab191703256e3e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:48.915565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:48.920470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:48.928167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 827/1667] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:48 2022] DELETE /identity/v3/projects/c9089fab4b8f48c4ab191703256e3e94 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:48.931349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:48.948111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a7d48-3f5e-4286-b436-bca102e21da4 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:49.061764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:49.062654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:49.111127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:49.111647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:49.112068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:49.112448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] PATH_INFO: `/v3/users/d9b3374628dc4ff4b928c77d022f41f9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:49.114066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b3374628dc4ff4b928c77d022f41f9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:49.116203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:49.224879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c17201-808b-49de-a9b6-0165b99aa5bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:49.239159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:49.239356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:49.239590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:49.240106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] PATH_INFO: `/v3/users/b226ddd101ae4ff9aeab0fc797a349f5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:49.242913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b226ddd101ae4ff9aeab0fc797a349f5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:49.244611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:49.435182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b3374628dc4ff4b928c77d022f41f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.436575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b3374628dc4ff4b928c77d022f41f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.450364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b3374628dc4ff4b928c77d022f41f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.536677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b226ddd101ae4ff9aeab0fc797a349f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.542647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b3374628dc4ff4b928c77d022f41f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.542931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b226ddd101ae4ff9aeab0fc797a349f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.553607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b3374628dc4ff4b928c77d022f41f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.568420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b226ddd101ae4ff9aeab0fc797a349f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.570937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 841/1668] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:48 2022] DELETE /identity/v3/users/d9b3374628dc4ff4b928c77d022f41f9 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:49.572028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:49.574428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:49.614666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b226ddd101ae4ff9aeab0fc797a349f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.624104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:49.683666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89b0c8-8dad-4f74-ac82-77f38f4abad6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:49.692502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:49.693609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:49.694101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:49.694695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] PATH_INFO: `/v3/projects/34c47c71c036488888c1bf5eba9d3855` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:49.696989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c47c71c036488888c1bf5eba9d3855)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:49.699277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:49.731401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a2e7c64-7f33-40ff-8b1c-21e7e0a654b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b226ddd101ae4ff9aeab0fc797a349f5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.743930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 828/1669] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:49 2022] DELETE /identity/v3/users/b226ddd101ae4ff9aeab0fc797a349f5 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:03:49.753749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:49.754329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:49.755383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:49.807504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:49.809147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:49.822369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:49.960228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c47c71c036488888c1bf5eba9d3855', 'request_id': 'req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:49.986313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c47c71c036488888c1bf5eba9d3855', 'request_id': 'req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.009156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 829/1670] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:50.066460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:50.068135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:50.199133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec6844-cf6d-4448-83f9-f6e353b8c392 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:50.233547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c47c71c036488888c1bf5eba9d3855', 'request_id': 'req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.431599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] The token cache is being invalidate because project 34c47c71c036488888c1bf5eba9d3855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:50.431599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.431599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.431599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 842/1671] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:49 2022] DELETE /identity/v3/projects/34c47c71c036488888c1bf5eba9d3855 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:50.432910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:50.432910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:50.432910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:50.432910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:50.433499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:50.434247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:50.434247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:50.456202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:50.460647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:50.494707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 830/1672] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:50.540176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce879b8d-5904-4d31-a79a-8f0c47f9f8b4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:50.556889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:50.556889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:50.556889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:50.556889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] PATH_INFO: `/v3/users/689d494727f54427aa20285445c3e647` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:50.557185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=689d494727f54427aa20285445c3e647)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:50.559814 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:50.840014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:50.842710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:50.895447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb4717-4fd7-4ee7-8ab2-1549b344dc34 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:50.901486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '689d494727f54427aa20285445c3e647'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.903522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '689d494727f54427aa20285445c3e647'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:50.930468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:50.931181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:50.931457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:50.935519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] PATH_INFO: `/v3/projects/704c7146830a4c58930737948274860e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:50.943076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=704c7146830a4c58930737948274860e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:50.955276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:50.969778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '689d494727f54427aa20285445c3e647'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.077897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '689d494727f54427aa20285445c3e647'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.090721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e90b35-43cc-4ece-8cb0-0b38c46c55cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '689d494727f54427aa20285445c3e647'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.147592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 843/1673] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:50 2022] DELETE /identity/v3/users/689d494727f54427aa20285445c3e647 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:51.270408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704c7146830a4c58930737948274860e', 'request_id': 'req-290eb50d-f9c0-40f8-b7fb-d13284717e15'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.287613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704c7146830a4c58930737948274860e', 'request_id': 'req-290eb50d-f9c0-40f8-b7fb-d13284717e15'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.416134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704c7146830a4c58930737948274860e', 'request_id': 'req-290eb50d-f9c0-40f8-b7fb-d13284717e15'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.467601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] The token cache is being invalidate because project 704c7146830a4c58930737948274860e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:51.468295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.472953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-290eb50d-f9c0-40f8-b7fb-d13284717e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:51.475267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 831/1674] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:50 2022] DELETE /identity/v3/projects/704c7146830a4c58930737948274860e => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:51.480121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fea59-8703-474e-97bf-45fd2838f4f6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.482978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fea59-8703-474e-97bf-45fd2838f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.483418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fea59-8703-474e-97bf-45fd2838f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.540986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152791c9-e0ca-41e8-95f0-c5e43ccaabc3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.541844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152791c9-e0ca-41e8-95f0-c5e43ccaabc3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.542302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152791c9-e0ca-41e8-95f0-c5e43ccaabc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.547500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-020fea59-8703-474e-97bf-45fd2838f4f6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:51.547955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-020fea59-8703-474e-97bf-45fd2838f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:51.646653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-152791c9-e0ca-41e8-95f0-c5e43ccaabc3 None None] MFA Rules not processed for user `2d6fc583bb8a4434b0c6538c9ffbde75`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:51.650340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-152791c9-e0ca-41e8-95f0-c5e43ccaabc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:51.737312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 844/1675] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:51.739279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.739873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.740350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.773087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:51.779235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:51.795707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 845/1676] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:51.796950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:51.810029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3aa7-acfa-4ba3-bc3d-fc10027afd97 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:51.816597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:51.817359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.817701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.820176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.820176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:51.820176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:51.822002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:51.842454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 846/1677] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:51.843846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:51.856788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fdaab-c3cc-412d-98bb-9615bda62350 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:51.862994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:51.864402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.864809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.865264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.866207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:51.868416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:51.893012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 832/1678] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:51.894619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:51.895553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:51.896126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:51.980937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] MFA Rules not processed for user `2d6fc583bb8a4434b0c6538c9ffbde75`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:51.982749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:52.004348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 847/1679] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:51 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:52.006599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.012489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 833/1680] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:52.020511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.021981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:52.183302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d65cc-e0d0-4f72-a423-c400d9993c09 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:52.194742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:52.196196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:52.196676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:52.197197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:52.198259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:52.201216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:52.353289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf130b-4721-4e0a-91a9-0751fcdf2ebf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:52.377721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 848/1681] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:52 2022] POST /identity/v3/users => generated 294 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:52.382372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.406889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f8f0d-2f67-40db-bd12-07d73102fd4e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:52.413560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:52.414892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:52.416322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:52.416670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:52.417912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:52.424065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:52.465358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 849/1682] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:52.466951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.469196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:52.492194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c69c1-4c47-47b8-96d3-ca318d3b3da3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:52.509685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:52.510925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:52.511243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:52.511483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] PATH_INFO: `/v3/projects/d11d73b282114c02bed69938677e2e58` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:52.512726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11d73b282114c02bed69938677e2e58)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:52.516341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:52.559727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:52.561440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:52.561739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:52.562156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:52.734949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11d73b282114c02bed69938677e2e58', 'request_id': 'req-71c7500c-33d0-49a8-9523-252f1acd5bd3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:52.764390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11d73b282114c02bed69938677e2e58', 'request_id': 'req-71c7500c-33d0-49a8-9523-252f1acd5bd3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:52.768645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:52.775873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:52.817149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:52.817380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 834/1683] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 797 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:52.825815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.848895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95620d1-f786-4a00-a43a-60af18e911eb service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:52.866896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11d73b282114c02bed69938677e2e58', 'request_id': 'req-71c7500c-33d0-49a8-9523-252f1acd5bd3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:52.906203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] The token cache is being invalidate because project d11d73b282114c02bed69938677e2e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:52.906587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:52.909170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:52.916307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 850/1684] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:52 2022] DELETE /identity/v3/projects/d11d73b282114c02bed69938677e2e58 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:52.925291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:52.931751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:53.002073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:53.002746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.003043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.003385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.004493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:53.006156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fa418-50c1-4589-9074-4d13f6fb8701 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:53.017217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:53.024263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 835/1685] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:53.025631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.025987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.026335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.115208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:53.120081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:53.147878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 836/1686] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:53.149161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:53.152510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:53.182477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ec9aa-a652-41eb-9637-80c0d79e8a20 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:53.194352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7500c-33d0-49a8-9523-252f1acd5bd3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:53.195803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:53.197213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.198712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.199084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] PATH_INFO: `/v3/projects/f285b88c2aa547ddb8444d5f2c2af5db/users/50dd741fa4e5417fab9f7919e1c19d66/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.212429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:53.213277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.213561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.213838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] PATH_INFO: `/v3/users/9350407241ea496fa23cf9a883cb2b04` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.215057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9350407241ea496fa23cf9a883cb2b04)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:53.217987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:53.238543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f285b88c2aa547ddb8444d5f2c2af5db, user_id=50dd741fa4e5417fab9f7919e1c19d66, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:53.242320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3def77-a760-448f-b96f-3a1c74e731dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:53.401548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 837/1687] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:53 2022] PUT /identity/v3/projects/f285b88c2aa547ddb8444d5f2c2af5db/users/50dd741fa4e5417fab9f7919e1c19d66/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:53.404727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc916-7ef0-43fb-8f1d-78c65ef4f4d3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.405109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc916-7ef0-43fb-8f1d-78c65ef4f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.405459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc916-7ef0-43fb-8f1d-78c65ef4f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.477756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-01bcc916-7ef0-43fb-8f1d-78c65ef4f4d3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:53.479367 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01bcc916-7ef0-43fb-8f1d-78c65ef4f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:53.568572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9350407241ea496fa23cf9a883cb2b04'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:53.568572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9350407241ea496fa23cf9a883cb2b04'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:53.634689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9350407241ea496fa23cf9a883cb2b04'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:53.678664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 838/1688] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:53.678664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.678664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.679013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.722849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] MFA Rules not processed for user `50dd741fa4e5417fab9f7919e1c19d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:53.724977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:53.736066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9350407241ea496fa23cf9a883cb2b04'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:53.757186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d36db58-8483-4f1a-a752-5b8e643a49fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9350407241ea496fa23cf9a883cb2b04'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:53.759251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:53.766584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 851/1689] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:52 2022] DELETE /identity/v3/users/9350407241ea496fa23cf9a883cb2b04 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:53.768286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:53.768914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:53.773342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:53.824224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] MFA Rules not processed for user `684d59041e82466d8aad6d2964842bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:53.828964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.010796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 839/1690] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.012347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.037661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf80be-ad7b-461b-8476-4a1853cc7548 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.079790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 852/1691] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.080975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.090920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.183680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:54.199235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3604-cee5-4f0f-ace9-76cd1ad3821c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:54.199971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731fe4-b1ec-4113-a15c-66e5f3957cfd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.199971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:54.199971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.200623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.200623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] PATH_INFO: `/v3/projects/3d93ea156d4240619162154e3864e012` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.201799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d93ea156d4240619162154e3864e012)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:54.203715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:54.207622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 840/1692] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:54.208707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421454-ea59-4d92-bfa5-83debb02936f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.209867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421454-ea59-4d92-bfa5-83debb02936f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.210237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14421454-ea59-4d92-bfa5-83debb02936f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.242225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-14421454-ea59-4d92-bfa5-83debb02936f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:54.243538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14421454-ea59-4d92-bfa5-83debb02936f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.253194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 841/1693] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.255073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.255687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.256146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.294119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] MFA Rules not processed for user `1b7724f3ef4543ed9c3df0455ae4fb76`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:54.298917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.389421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 842/1694] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.389421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.389421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.405162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568a46d-a31d-443b-a5be-79052a89314a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.418581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d93ea156d4240619162154e3864e012', 'request_id': 'req-681421ef-a051-4e76-9200-e96666f342f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:54.448327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d93ea156d4240619162154e3864e012', 'request_id': 'req-681421ef-a051-4e76-9200-e96666f342f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:54.479091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:54.484389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.484389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.484389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.484389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:54.484389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a332f1-84b3-4a61-8437-46492d2c21bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:54.571009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:54.571696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d93ea156d4240619162154e3864e012', 'request_id': 'req-681421ef-a051-4e76-9200-e96666f342f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:54.585681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] The token cache is being invalidate because project 3d93ea156d4240619162154e3864e012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:54.586009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:54.586803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-681421ef-a051-4e76-9200-e96666f342f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:54.588421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 853/1695] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:54 2022] DELETE /identity/v3/projects/3d93ea156d4240619162154e3864e012 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:54.595829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.596766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.596766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.620659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] MFA Rules not processed for user `684d59041e82466d8aad6d2964842bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:54.621863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.643324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 843/1696] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/projects => generated 385 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:54.649067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.649067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.649067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.649067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 844/1697] 10.222.0.48 () {54 vars in 912 bytes} [Thu Aug 11 11:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:54.655135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.657019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.696356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 854/1698] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.697527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.698966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.739647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fdbbb-3447-4cc5-b6f4-3c8ed2db81c9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.744308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:54.744967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.745177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.745448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.746071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:54.748055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8988170-63a5-4d4d-98a7-fb77f4dcfaa3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:54.760398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5a256-08e5-43c6-859b-bbf660e2d729 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.811237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:54.812447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.812725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.812991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.815073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.827331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:54.828909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:54.840406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 855/1699] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:54.844233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 845/1700] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:54.845076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:54.846653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02075-6766-4b1b-bf0d-f43b229a8eed None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.846971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02075-6766-4b1b-bf0d-f43b229a8eed None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.847245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b02075-6766-4b1b-bf0d-f43b229a8eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.851226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.874538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-44b02075-6766-4b1b-bf0d-f43b229a8eed None None] MFA Rules not processed for user `bb18a00d44ed4af186bdc589e1e14e22`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:54.876344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44b02075-6766-4b1b-bf0d-f43b229a8eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:54.947666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 846/1701] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:54.950141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:54.951895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:54.956967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:54.961755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183ef75-7df4-4dde-87f3-e9c03c27a7a8 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:54.980777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:54.982090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.026682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.027319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.027598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.027918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.030097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.040296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 847/1702] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:55.041339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.042575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.053918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b63cb3-392a-49a6-8b6c-db7d6092a036 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.058409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.058991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.059266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.059542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.060983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.062880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.089056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.094333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19a6b0-e10d-4150-8693-9552aa22d686 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.100025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 856/1703] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:55.101382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcfff-bc8c-4318-864e-d8671dc4e5d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.101785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcfff-bc8c-4318-864e-d8671dc4e5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.102061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcfff-bc8c-4318-864e-d8671dc4e5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.153753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cfebcfff-bc8c-4318-864e-d8671dc4e5d5 None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:55.155052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cfebcfff-bc8c-4318-864e-d8671dc4e5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.177060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 848/1704] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:55 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:55.178179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.203681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c2545-ec33-4466-8887-0b990b172c55 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.211509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.212527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.212912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.215795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.217174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.224641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.303294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 857/1705] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:03:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:55.305763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.306289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.309833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.322975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 849/1706] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:55 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:55.324110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.336652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:55.339506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.340330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fccf7-b2b9-4d2c-a5b9-c777b2a0ca20 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.347085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.347647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.348547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.348839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.349557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.352111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.360150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 858/1707] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:55.360150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.367218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 850/1708] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:55.368372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.372866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7ed85-b9dc-407e-9430-75ae335f76eb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.381053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612c68e-66ed-4050-8fb6-b190759dfb94 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.384025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.384650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.384970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.385201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.392834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.407663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.407663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.407663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.407663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.408076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.408164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.408251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.408251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 859/1709] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:55.408251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.408486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 851/1710] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:55.408917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.418926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185a6d9-2385-4bb9-803b-51da693e37c9 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.423091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d6c1-8dc1-43df-909b-1e1fbffc94ad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.428090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.428706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.428841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.429179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.429521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.429638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.429985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.430343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] PATH_INFO: `/v3/projects/31e04290400e4729bb070cfd8999e24d/users/b63c980129c84d35850100e802bd4446/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.430459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:55.442799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.447140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.449987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e04290400e4729bb070cfd8999e24d, user_id=b63c980129c84d35850100e802bd4446, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.454596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.503100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 860/1711] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:55.509225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.535254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5b7c-079a-41ae-80d7-2b544b0d884d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.546658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.547421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.547754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.548128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.550924 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.580723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 852/1712] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:55 2022] PUT /identity/v3/projects/31e04290400e4729bb070cfd8999e24d/users/b63c980129c84d35850100e802bd4446/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:55.580723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.701700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:55.710172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.712413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.724925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 861/1713] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:55.726067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.728043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:55.751798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41cc1d-e1c5-4a88-bc32-7e17b4851e76 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.758211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1142f8-1ae4-4ff9-856c-05c53b168141 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.759942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.760669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.761028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.761397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] PATH_INFO: `/v3/projects/74eff4ac3477460cb61a800a3a1add39` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.763262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74eff4ac3477460cb61a800a3a1add39)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.765631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.773161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.773161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.773161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.773440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.773534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.773534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.786677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 853/1714] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:55.787933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:55.854897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867984-5a07-4b26-879b-cbd60251f464 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:55.877486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:55.877486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:55.877486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:55.877486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:55.877913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:55.878209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:55.941568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74eff4ac3477460cb61a800a3a1add39', 'request_id': 'req-f7dc604f-1d49-4a82-9da2-82dc06addd05'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:55.960197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74eff4ac3477460cb61a800a3a1add39', 'request_id': 'req-f7dc604f-1d49-4a82-9da2-82dc06addd05'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:56.150993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74eff4ac3477460cb61a800a3a1add39', 'request_id': 'req-f7dc604f-1d49-4a82-9da2-82dc06addd05'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:56.151915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 854/1715] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:55 2022] POST /identity/v3/projects => generated 369 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:56.152859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.159925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] The token cache is being invalidate because project 74eff4ac3477460cb61a800a3a1add39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:03:56.160651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:56.163809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:56.172243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 862/1716] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:03:55 2022] DELETE /identity/v3/projects/74eff4ac3477460cb61a800a3a1add39 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:56.172243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.173507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.250679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7e263-8330-4c97-99d9-a7ac22ff99e6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:56.257548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:56.259020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.259599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.260045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.261137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:56.264365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:56.291568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc604f-1d49-4a82-9da2-82dc06addd05 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:56.291695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 855/1717] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:56.293471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.293771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:56.295391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.295679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.295941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] PATH_INFO: `/v3/projects/de459d639db84999940b6b3f56611fa8/users/70d60c3f19ce4d0f8fb4646ee3b32888/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.296659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.328363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de459d639db84999940b6b3f56611fa8, user_id=70d60c3f19ce4d0f8fb4646ee3b32888, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:56.351996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eee13-a68e-4735-a6bc-39532970090f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:56.354847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebce24-91ad-4613-a4b3-8a04089bb30f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:56.368475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:56.369699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.381250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.381250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.381250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:56.381250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:56.465019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 863/1718] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:56 2022] PUT /identity/v3/projects/de459d639db84999940b6b3f56611fa8/users/70d60c3f19ce4d0f8fb4646ee3b32888/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:56.467325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7220436-05cd-47f7-9608-7ce1c15aa7bc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.467852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7220436-05cd-47f7-9608-7ce1c15aa7bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.468373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7220436-05cd-47f7-9608-7ce1c15aa7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.491287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 856/1719] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:56.498361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.500949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.519472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a7220436-05cd-47f7-9608-7ce1c15aa7bc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:56.521330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a7220436-05cd-47f7-9608-7ce1c15aa7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.530098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae279c7-0d4f-4853-aaae-5c1e795e1016 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:56.640526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 864/1720] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:56.641838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78a27a-ce3b-4ea6-b954-f9d83b10ecf8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.642224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78a27a-ce3b-4ea6-b954-f9d83b10ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.642573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78a27a-ce3b-4ea6-b954-f9d83b10ecf8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.643774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 865/1721] 10.222.0.48 () {54 vars in 912 bytes} [Thu Aug 11 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:56.644782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951068a-2a79-4491-8b27-82aa08d43596 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.645065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951068a-2a79-4491-8b27-82aa08d43596 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.645339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951068a-2a79-4491-8b27-82aa08d43596 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.647005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 866/1722] 10.222.0.48 () {54 vars in 961 bytes} [Thu Aug 11 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:03:56.648409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5855-7c08-4896-9383-a9e1a1bd6a03 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.648692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5855-7c08-4896-9383-a9e1a1bd6a03 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.649037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf5855-7c08-4896-9383-a9e1a1bd6a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.656576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:56.658887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.659388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.659862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] PATH_INFO: `/v3/projects/31e04290400e4729bb070cfd8999e24d/users/b63c980129c84d35850100e802bd4446/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.662244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e04290400e4729bb070cfd8999e24d, user_id=b63c980129c84d35850100e802bd4446, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:56.666421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:56.688316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 857/1723] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:56 2022] PUT /identity/v3/projects/31e04290400e4729bb070cfd8999e24d/users/b63c980129c84d35850100e802bd4446/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:56.689612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.694006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-90bf5855-7c08-4896-9383-a9e1a1bd6a03 None None] MFA Rules not processed for user `70d60c3f19ce4d0f8fb4646ee3b32888`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:56.696781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-90bf5855-7c08-4896-9383-a9e1a1bd6a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.777471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29ea0c-b926-4798-beb3-c2f9729b0b03 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:56.784328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:56.785260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.785760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.785994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 867/1724] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:56.787261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89038e32-98d9-492c-b3d8-0034526b1cef None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.787543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89038e32-98d9-492c-b3d8-0034526b1cef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.787843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89038e32-98d9-492c-b3d8-0034526b1cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.789748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.790529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:56.792329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082d5f1-cb32-48f8-b827-6896d73b90cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:56.810683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 858/1725] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:56.813325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47bbc6-67ea-483a-af52-2f51fc714b4e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.813325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47bbc6-67ea-483a-af52-2f51fc714b4e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.813325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47bbc6-67ea-483a-af52-2f51fc714b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.815892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-89038e32-98d9-492c-b3d8-0034526b1cef None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:56.820468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89038e32-98d9-492c-b3d8-0034526b1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.836122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab47bbc6-67ea-483a-af52-2f51fc714b4e None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:56.837697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab47bbc6-67ea-483a-af52-2f51fc714b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.847954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 868/1726] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:56.847954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.847954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.847954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.870283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:56.874574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:56.874574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:56.941504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 859/1727] 10.222.0.48 () {58 vars in 1058 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:56.949825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:56.950924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:56.958644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:56.962423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 869/1728] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:56.963407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:56.988484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76eee-dfc1-46dd-aa9a-91bf76d9fd0d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.003142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.003729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.004038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.004406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.004935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.006846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.020845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] MFA Rules not processed for user `b63c980129c84d35850100e802bd4446`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:57.022247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:57.127311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 870/1729] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:57.127311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.200180 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 860/1730] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:57.219176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feb132-3df0-491d-9920-f950d76fba04 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.223735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.227579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.229052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.229347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.229620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.232953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.236719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.238755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526f5a1-584f-4b14-add3-84e1a6cac256 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.244783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.246024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.246413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.246967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.247907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:03:57.248284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.250044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.250683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 871/1731] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:57.251540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.285282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 861/1732] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:03:57.286618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:57.288791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.291368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:57.383571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530576e-f93f-4b28-ae44-ced54930bae5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.394564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.394564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.394564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.394564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.396224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:57.402311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634980f-3a8b-479f-91e9-daee5d79be05 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.408439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.409319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.409743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.410964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.422740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.423106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.424541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.424773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.434142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 862/1733] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:57.434875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.457776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 872/1734] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:57.458671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.468010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:57.485043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d07f8-c91d-4323-9bad-f516c3156698 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.511235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.516349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.516706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.516989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.517664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.527545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.586835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f6a0-e7fa-4400-bb01-297712e08548 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.601278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.606753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.607244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.609784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.646295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 863/1735] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:57 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:57.651224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.697510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767502f8-1423-4ee5-a16b-a60dded73dff admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:57.706747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:57.708656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:57.708656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:57.708656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] PATH_INFO: `/v3/projects/c6305df0f06d4f2b96ac02bb7a7ebaf8/users/85498677028f454291e1f2ed687e2c4b/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:57.779631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6305df0f06d4f2b96ac02bb7a7ebaf8, user_id=85498677028f454291e1f2ed687e2c4b, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.795028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.867754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:57.879381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:57.916182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 873/1736] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:03:57.921140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.942216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 864/1737] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:57 2022] PUT /identity/v3/projects/c6305df0f06d4f2b96ac02bb7a7ebaf8/users/85498677028f454291e1f2ed687e2c4b/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:57.942932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:57.987755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2b0a8-cc98-4c38-a01e-01534d4f81e5 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:58.116216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:58.227736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b244a9-2cfd-4702-9afa-f11760f16292 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:58.257773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:58.258403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:58.258688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:58.258960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:58.270216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:58.278857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12371f10-f218-45e7-b735-6ee94e010aff admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:58.304736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:58.305371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:58.305659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:58.305926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:58.313308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:58.318662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:58.320676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 865/1738] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:58.322065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:58.322355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:58.323924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:58.372395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] MFA Rules not processed for user `85498677028f454291e1f2ed687e2c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:58.373633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:58.435114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 874/1739] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:57 2022] POST /identity/v3/projects => generated 371 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:58.436320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:58.530596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 866/1740] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:58.530596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:58.557485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14f241-7b55-4ba5-a135-09234b319d09 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:58.574904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9ce3-efd6-4622-bba0-4ec43e938c9b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:58.587420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:58.588065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:58.588725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:58.588725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] PATH_INFO: `/v3/users/50dd741fa4e5417fab9f7919e1c19d66` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:58.609666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dd741fa4e5417fab9f7919e1c19d66)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:58.621614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:58.622782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:58.623303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:58.623697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:58.624066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] PATH_INFO: `/v3/projects/2e778d9bdec04c009c2e7b3d12514538/users/326583e4dd1c4ce2b50e9d075dbbf649/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:58.657409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e778d9bdec04c009c2e7b3d12514538, user_id=326583e4dd1c4ce2b50e9d075dbbf649, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:58.660584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:58.808717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 867/1741] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:58 2022] PUT /identity/v3/projects/2e778d9bdec04c009c2e7b3d12514538/users/326583e4dd1c4ce2b50e9d075dbbf649/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:58.808717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:58.920538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dd741fa4e5417fab9f7919e1c19d66'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:58.921368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dd741fa4e5417fab9f7919e1c19d66'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:58.976971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dd741fa4e5417fab9f7919e1c19d66'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:59.103831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dd741fa4e5417fab9f7919e1c19d66'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:59.110427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dd741fa4e5417fab9f7919e1c19d66'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:03:59.115377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ae5b6-dac1-4c42-acee-fbbcff0e4bed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.121662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 875/1742] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:03:58 2022] DELETE /identity/v3/users/50dd741fa4e5417fab9f7919e1c19d66 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:59.123030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.135710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.255236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260ffa3-fadf-4020-81f5-7c603a396035 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.260570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.261469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.261753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.262044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.262700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.265070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999573-312a-46b4-b192-a1e5134e8086 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.293135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.294451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.294826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.295173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.295871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.297816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.445418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 868/1743] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:58 2022] POST /identity/v3/users => generated 286 bytes in 636 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:59.445418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.450473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 876/1744] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:03:59 2022] POST /identity/v3/projects => generated 369 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:59.451313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3419-2e92-466b-a41c-d7bd326a20e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.451739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3419-2e92-466b-a41c-d7bd326a20e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.452096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3419-2e92-466b-a41c-d7bd326a20e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.455472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.477055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe53a9-4fc7-4eff-bff0-8941937094df admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.478704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6e9a3419-2e92-466b-a41c-d7bd326a20e4 None None] MFA Rules not processed for user `684d59041e82466d8aad6d2964842bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:59.480168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e9a3419-2e92-466b-a41c-d7bd326a20e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.621209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 877/1745] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:59.623749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.624232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.630080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.630835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.631565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.631728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.636413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.637937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.641467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.684160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:03:59.684160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.686049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 869/1746] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:59.687320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.689257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.701692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 878/1747] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:03:59.701692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.709038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb4f56-e2d6-4871-b482-fbe913318d0a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.716133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.716133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.716133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.732333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef48b1-6820-4086-a8e0-f552420ab934 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.738425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.739979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.743151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.750802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:03:59.764951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.766942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.768190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.769406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.771198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.781219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.782526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 870/1748] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:03:59.784557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.813902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0384c5-ff9d-4cc1-a476-6b5a4e8a15aa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.820607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.821885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.822366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.823028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] PATH_INFO: `/v3/projects/2e778d9bdec04c009c2e7b3d12514538/users/326583e4dd1c4ce2b50e9d075dbbf649/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:03:59.826782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e778d9bdec04c009c2e7b3d12514538, user_id=326583e4dd1c4ce2b50e9d075dbbf649, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:03:59.836858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:03:59.844020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 871/1749] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:03:59 2022] PUT /identity/v3/projects/2e778d9bdec04c009c2e7b3d12514538/users/326583e4dd1c4ce2b50e9d075dbbf649/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:03:59.845190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.847093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:03:59.918491 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 879/1750] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:03:59 2022] POST /identity/v3/users => generated 291 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:03:59.919564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:03:59.944337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6132c-38f2-4263-82f6-96c84dad7922 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.946395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae981f6-4c35-4859-b5d7-d26806d3fabd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:03:59.952041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:03:59.952908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:03:59.953207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:03:59.953589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.067929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:00.070058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14452e51-f42b-4edf-92c1-02bdd1fd4d9d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:00.091330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 872/1751] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:00.093413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:00.094360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1de76-ade1-4d12-9b83-0eccd9d6185b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.098218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1de76-ade1-4d12-9b83-0eccd9d6185b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.098218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1de76-ade1-4d12-9b83-0eccd9d6185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.098916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.098916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.098916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.098916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:00.114855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:00.158472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 880/1752] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:00.158472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:00.189027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756a17-5a83-42b8-a59d-d0519e66bf1d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:00.197660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-21c1de76-ade1-4d12-9b83-0eccd9d6185b None None] MFA Rules not processed for user `326583e4dd1c4ce2b50e9d075dbbf649`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:00.198646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21c1de76-ade1-4d12-9b83-0eccd9d6185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:00.203603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:00.204657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.204947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.213189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.213189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:00.213189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:00.244800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 881/1753] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:00.245667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:00.280653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc1d96-2eac-471a-8594-d321c1f40c35 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:00.292628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:00.295461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.295947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.296229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] PATH_INFO: `/v3/projects/ada4dc9f9ffe430cba190934fe06f0c6/users/4d1875f1354e4c9eb011e7ede97e1745/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.334488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ada4dc9f9ffe430cba190934fe06f0c6, user_id=4d1875f1354e4c9eb011e7ede97e1745, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:00.342764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:00.369687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 873/1754] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:00.371014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.371277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.371559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.393819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] MFA Rules not processed for user `1b7724f3ef4543ed9c3df0455ae4fb76`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:00.395254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:00.437768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:00.469278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 882/1755] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:00 2022] PUT /identity/v3/projects/ada4dc9f9ffe430cba190934fe06f0c6/users/4d1875f1354e4c9eb011e7ede97e1745/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:00.469991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:00.525688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 874/1756] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:00.526933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:00.528474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:00.667237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ec0e5-e2cd-4ff7-b160-8dd0fbbdbfa5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:00.727557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17682fdf-fb46-4c6d-8f48-4a7f810db3a8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:00.773849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:00.774714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.775003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.777002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] PATH_INFO: `/v3/projects/50bdc9a2b0854adda828bbf019797ac3/users/7dd71feab7b2458d8fb7e46f0374c5ef/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.810764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bdc9a2b0854adda828bbf019797ac3, user_id=7dd71feab7b2458d8fb7e46f0374c5ef, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:00.812453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d96f6-b2a9-4bfd-ab52-3e96a0361420 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:00.904376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:00.908364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.908578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.908861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:00.994254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 883/1757] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:00 2022] PUT /identity/v3/projects/50bdc9a2b0854adda828bbf019797ac3/users/7dd71feab7b2458d8fb7e46f0374c5ef/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:00.995773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:00.996125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:00.997514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:01.092676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] MFA Rules not processed for user `4d1875f1354e4c9eb011e7ede97e1745`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:01.094919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:01.155562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:01.160355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:01.206960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 875/1758] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:01.207978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:01.237047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddefc4c-6c1c-4e25-92f5-bb97271a5a8c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:01.385124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 884/1759] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:01.386226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:01.389433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:01.540729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:01.541348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:01.547473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:01.548520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:01.550366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:01.554555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:01.574204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 876/1760] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:01.575710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:01.593673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84f95d-4360-44e7-8450-b0f071e94034 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:01.604471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:01.605650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:01.606685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:01.607279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:01.608169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:01.612284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:01.641815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a6e70-664b-4269-b711-1616e045e505 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:01.658916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:01.659875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:01.660761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:01.663681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] PATH_INFO: `/v3/projects/f285b88c2aa547ddb8444d5f2c2af5db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:01.666354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f285b88c2aa547ddb8444d5f2c2af5db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:01.671085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:01.707591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 877/1761] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:01 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:01.708898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:01.893657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c2fd7-c0f4-450d-adc4-ed27e1e9704f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:01.894015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f285b88c2aa547ddb8444d5f2c2af5db', 'request_id': 'req-37cbc802-9a20-486f-88e4-4731261e0df7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:01.932390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f285b88c2aa547ddb8444d5f2c2af5db', 'request_id': 'req-37cbc802-9a20-486f-88e4-4731261e0df7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.030607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.031395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.031685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.032035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] PATH_INFO: `/v3/users/70d60c3f19ce4d0f8fb4646ee3b32888` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.036524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d60c3f19ce4d0f8fb4646ee3b32888)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.038500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.107427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f285b88c2aa547ddb8444d5f2c2af5db', 'request_id': 'req-37cbc802-9a20-486f-88e4-4731261e0df7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.145822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] The token cache is being invalidate because project f285b88c2aa547ddb8444d5f2c2af5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:02.146256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.148122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.150981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 885/1762] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:01 2022] DELETE /identity/v3/projects/f285b88c2aa547ddb8444d5f2c2af5db => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:02.151974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.153831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.177536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbc802-9a20-486f-88e4-4731261e0df7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.183038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.185606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.186046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.188386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] PATH_INFO: `/v3/projects/50bdc9a2b0854adda828bbf019797ac3/users/7dd71feab7b2458d8fb7e46f0374c5ef/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.191146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50bdc9a2b0854adda828bbf019797ac3, user_id=7dd71feab7b2458d8fb7e46f0374c5ef, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.195930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.217350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 886/1763] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:02 2022] PUT /identity/v3/projects/50bdc9a2b0854adda828bbf019797ac3/users/7dd71feab7b2458d8fb7e46f0374c5ef/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:02.217350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.217816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.318767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d60c3f19ce4d0f8fb4646ee3b32888'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.320136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d60c3f19ce4d0f8fb4646ee3b32888'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.336604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d60c3f19ce4d0f8fb4646ee3b32888'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.350121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d60c3f19ce4d0f8fb4646ee3b32888'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.359410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542ffc-5b3b-4fc3-a531-819709833826 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.364259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.364955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.365299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.365641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.366323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.368357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.502232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-166a37e9-a481-40a1-afcb-1e808241a35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d60c3f19ce4d0f8fb4646ee3b32888'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.511654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 887/1764] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:02 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:02.513326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 878/1765] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:01 2022] DELETE /identity/v3/users/70d60c3f19ce4d0f8fb4646ee3b32888 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:02.513466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.513899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.514215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.514545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.515731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.531137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc436f-c709-48f3-a985-78c69f4e4c20 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.601742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.602851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.602851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.603364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] PATH_INFO: `/v3/users/b63c980129c84d35850100e802bd4446` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.604371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63c980129c84d35850100e802bd4446)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.607550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.611728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] MFA Rules not processed for user `7dd71feab7b2458d8fb7e46f0374c5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:02.613820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.650068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:02.697179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 879/1766] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:02.698525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.710080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.728395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b297ae-7843-4e3d-a093-7a45bd11da99 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.733804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.734430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.734711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.735085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.736422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.738138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb89226-4eb4-472c-95b8-f5bb4b9fdb83 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.751949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 880/1767] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:02.753663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.753954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.754304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.774183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:02.775400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.784510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 881/1768] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:02.785407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.786657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:02.878864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d938e-b92d-444f-9c72-2dc6b0b3bca4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.882617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.885801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.885801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.885801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.895122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.896672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.911022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 882/1769] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:02.921267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:02.933450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63c980129c84d35850100e802bd4446'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.935715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63c980129c84d35850100e802bd4446'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:02.943253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859030d9-c505-4c3d-93be-4187ebd11093 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:02.947933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:02.948595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:02.948878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:02.949319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] PATH_INFO: `/v3/projects/f609719885454622bc58c8e139a96bbe/users/c24a61479d75455c9fec63658f9a017e/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:02.966280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f609719885454622bc58c8e139a96bbe, user_id=c24a61479d75455c9fec63658f9a017e, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:02.969100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:02.988015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63c980129c84d35850100e802bd4446'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:03.011558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:03.074105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 883/1770] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:02 2022] PUT /identity/v3/projects/f609719885454622bc58c8e139a96bbe/users/c24a61479d75455c9fec63658f9a017e/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:03.074783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:03.076710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.100996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63c980129c84d35850100e802bd4446'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:03.110111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63c980129c84d35850100e802bd4446'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:03.118784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 888/1771] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:02 2022] DELETE /identity/v3/users/b63c980129c84d35850100e802bd4446 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:03.120013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:03.122285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.176843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb8c6-f453-4a4a-98db-6fb5cab7427c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:03.249658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2074ae-205f-43c5-9feb-e70d46c6a4e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:03.255741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:03.256703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.257212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.257663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.258483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:03.263120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:03.276376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:03.277198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.278301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.278835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.287448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.325809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 889/1772] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:03.328125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:03.328246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:03.332266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.332543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:03.361197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 884/1773] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:03.362665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:03.414432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771cc54-39c1-4655-9454-2b2af077ba50 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:03.442220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:03.442930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.443220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.443502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] PATH_INFO: `/v3/projects/f609719885454622bc58c8e139a96bbe/users/c24a61479d75455c9fec63658f9a017e/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.461605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f609719885454622bc58c8e139a96bbe, user_id=c24a61479d75455c9fec63658f9a017e, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:03.472579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f6dc5-596b-4a71-9d92-d25355bea6bc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:03.550775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 885/1774] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:03 2022] PUT /identity/v3/projects/f609719885454622bc58c8e139a96bbe/users/c24a61479d75455c9fec63658f9a017e/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:03.557933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.567664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.568141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.600463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46cf1-584b-40c0-8367-3b526d00718a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:03.666807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] MFA Rules not processed for user `c24a61479d75455c9fec63658f9a017e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:03.671271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.727236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:03.728023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.728333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.728675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.731494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.775377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 886/1775] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:03.787544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:03.788986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:03.816132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d414102-3555-4b6d-b0d9-fa6fc776525b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:03.825232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:03.825859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.826144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.826415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:03.845056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:03.846697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:03.846820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa5707-4b9d-4841-8272-cd3726907906 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:03.848472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:03.862016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 890/1776] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:03.890099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 887/1777] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:03.957689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:03.958204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:03.963295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:04.042048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:04.049626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:04.160813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:04.181846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 891/1778] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:04:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:04.676800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:04.678419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:04.825756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:04.827268 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:04.929752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744d85a-11d8-44c5-8657-42692e8cbba0 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:04.937522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:04.938517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:04.939198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:04.939792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:04.952409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:04.956603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5861bf6-548b-4bcc-9755-e189b918e02a service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:04.985370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 888/1779] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:04:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:04:05.115279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af749bc-68ed-43e9-92fa-a4b4cb5289c4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:05.138357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:05.139242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:05.144606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:05.144780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] PATH_INFO: `/v3/projects/31e04290400e4729bb070cfd8999e24d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:05.150304 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e04290400e4729bb070cfd8999e24d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:05.163060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:05.172732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:05.173362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:05.173741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:05.254945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:05.260353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:05.457663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 889/1780] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Aug 11 11:04:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:05.459215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e04290400e4729bb070cfd8999e24d', 'request_id': 'req-847f7b19-7911-42d8-a7cb-87f705931616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:05.480040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:05.483664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:05.490759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e04290400e4729bb070cfd8999e24d', 'request_id': 'req-847f7b19-7911-42d8-a7cb-87f705931616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:05.516790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cb97a-1e8b-4399-bb00-3fadedd3e3e2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:05.666385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e04290400e4729bb070cfd8999e24d', 'request_id': 'req-847f7b19-7911-42d8-a7cb-87f705931616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:05.674675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:05.690683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] The token cache is being invalidate because project 31e04290400e4729bb070cfd8999e24d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:05.690683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:05.694966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:05.695484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:05.696521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:05.697129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:05.697665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:05.702412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:05.718919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:05.723426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca02cd-276d-493e-aa1b-6aa3930d7ceb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:05.729999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 892/1781] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:04 2022] DELETE /identity/v3/projects/31e04290400e4729bb070cfd8999e24d => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:05.736317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:05.744693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:05.773554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 890/1782] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:05.775785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:05.776420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:05.776861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:05.822330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:05.827092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:05.954495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:06.062375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 891/1783] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:06.063295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:06.068640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:06.102028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7b19-7911-42d8-a7cb-87f705931616 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:06.110507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52d0c4-56c1-4078-ba41-6061a7271baa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:06.117812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:06.118228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:06.118315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:06.118527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] PATH_INFO: `/v3/projects/de459d639db84999940b6b3f56611fa8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:06.120617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de459d639db84999940b6b3f56611fa8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:06.125191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:06.126103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:06.127316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:06.130342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:06.130776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] PATH_INFO: `/v3/users/4d1875f1354e4c9eb011e7ede97e1745` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:06.136922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1875f1354e4c9eb011e7ede97e1745)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:06.141744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:06.374958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de459d639db84999940b6b3f56611fa8', 'request_id': 'req-9628d7ae-02ff-4a99-997e-792e20e0dcb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.395180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de459d639db84999940b6b3f56611fa8', 'request_id': 'req-9628d7ae-02ff-4a99-997e-792e20e0dcb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.518313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de459d639db84999940b6b3f56611fa8', 'request_id': 'req-9628d7ae-02ff-4a99-997e-792e20e0dcb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.537879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] The token cache is being invalidate because project de459d639db84999940b6b3f56611fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:06.538191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.541656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1875f1354e4c9eb011e7ede97e1745'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.542769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9628d7ae-02ff-4a99-997e-792e20e0dcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.545332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1875f1354e4c9eb011e7ede97e1745'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.548597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 893/1784] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:05 2022] DELETE /identity/v3/projects/de459d639db84999940b6b3f56611fa8 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:06.555709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f77e28-954a-401f-923f-510c4c5797b0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:06.556386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f77e28-954a-401f-923f-510c4c5797b0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:06.556909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f77e28-954a-401f-923f-510c4c5797b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:06.570744 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1875f1354e4c9eb011e7ede97e1745'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.588625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1875f1354e4c9eb011e7ede97e1745'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.644080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-93f77e28-954a-401f-923f-510c4c5797b0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:06.650076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-93f77e28-954a-401f-923f-510c4c5797b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:06.735207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1875f1354e4c9eb011e7ede97e1745'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:06.738018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:06.744880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 892/1785] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:06 2022] DELETE /identity/v3/users/4d1875f1354e4c9eb011e7ede97e1745 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:06.744880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:06.755674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:06.858774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 894/1786] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:06.863392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:06.864740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:06.865240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:06.896677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:06.898484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:06.987066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9da0f5-06e9-4b2e-b67d-084cce6e6bf9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:07.008591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:07.009504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:07.010124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:07.010858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] PATH_INFO: `/v3/users/326583e4dd1c4ce2b50e9d075dbbf649` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:07.013983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=326583e4dd1c4ce2b50e9d075dbbf649)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:07.016134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:07.208357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 895/1787] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:07.208357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:07.208357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85dccf-c336-4f21-acab-8d49071d77d6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:07.208357 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:07.211018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:07.211720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 896/1788] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:07.213297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:07.246084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702860ac-21a1-488f-9604-c66a1ed235c6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:07.266213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:07.266884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:07.267168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:07.267445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:07.268134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:07.276472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:07.549594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 897/1789] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:07 2022] POST /identity/v3/projects => generated 387 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:04:07.549594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:07.555796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:07.662276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326583e4dd1c4ce2b50e9d075dbbf649'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:07.663226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326583e4dd1c4ce2b50e9d075dbbf649'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:07.677486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326583e4dd1c4ce2b50e9d075dbbf649'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:07.704258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326583e4dd1c4ce2b50e9d075dbbf649'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:07.720322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a55d-095b-423f-b0af-d6d19804f85c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:07.745518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:07.747220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:07.747673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:07.756720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:07.763396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:07.842279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326583e4dd1c4ce2b50e9d075dbbf649'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:07.858512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 893/1790] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:06 2022] DELETE /identity/v3/users/326583e4dd1c4ce2b50e9d075dbbf649 => generated 0 bytes in 1114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:07.860002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:07.865602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:07.924895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:07.931546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:07.986655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 898/1791] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:08.063429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f68c36-b752-4613-a836-d9054e4c2cbb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:08.071070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:08.072133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:08.072619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:08.073082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:08.074698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:08.077944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:08.243326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 894/1792] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:07 2022] POST /identity/v3/users => generated 294 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:08.249563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:08.263723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ca30-cd82-41fc-91de-78e9117060a8 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:08.295278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:08.296047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:08.296426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:08.296928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:08.297931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:08.301231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:08.357373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 899/1793] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:08.359049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:08.376219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af8f3b-5bec-4846-8148-75e77ea3ca81 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:08.389311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:08.390753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:08.391354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:08.391877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:08.392758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:08.395948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7930d7-be05-40fd-9e76-190096a5d84b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:08.414398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 895/1794] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:08.415806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:08.428891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8282-ae18-4ede-a790-6104cb523d2b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:08.442352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:08.443544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:08.444059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:08.444896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] PATH_INFO: `/v3/projects/29ed54b31724454f9d35f24a9675b6c6/users/edf0eb65a3fa4f189d012d944c8096eb/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:08.489855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ed54b31724454f9d35f24a9675b6c6, user_id=edf0eb65a3fa4f189d012d944c8096eb, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:08.496991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:08.624346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 900/1795] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:08 2022] PUT /identity/v3/projects/29ed54b31724454f9d35f24a9675b6c6/users/edf0eb65a3fa4f189d012d944c8096eb/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:08.636166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c123653-212e-448f-be33-e62795eeb539 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:08.636561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c123653-212e-448f-be33-e62795eeb539 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:08.636861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c123653-212e-448f-be33-e62795eeb539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:08.701481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c123653-212e-448f-be33-e62795eeb539 None None] MFA Rules not processed for user `edf0eb65a3fa4f189d012d944c8096eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:08.702751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c123653-212e-448f-be33-e62795eeb539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:08.893712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 896/1796] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:08.894966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:09.023212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8277-dfb0-47c5-b0d7-7549d9d96c70 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:09.031416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:09.032099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.032383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.032684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.033292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:09.046408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:09.182614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 901/1797] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:08 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:09.187288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c123653-212e-448f-be33-e62795eeb539 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:09.203597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c123653-212e-448f-be33-e62795eeb539 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:09.348095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:09.349649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.350121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.350541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.351305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:09.354754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:09.369078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:09.497177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 897/1798] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:09 2022] POST /identity/v3/users => generated 300 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:09.498800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:09.558000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5f53b-7bdd-44d4-874d-08926f22bc9f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:09.576195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:09.576195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.576983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.577271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.577945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:09.583606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:09.601234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:09.609065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 902/1799] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:09.613877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:09.625875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f5e11-3649-4b13-a224-e415bb223933 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:09.643292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:09.646629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.646629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.646629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] PATH_INFO: `/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a/users/93c728bf1bb9407bbd612e3e0138f43c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.695798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e38a869e4d42d29a78aa3f93af7f4a, user_id=93c728bf1bb9407bbd612e3e0138f43c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:09.700000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b7edf-6b85-4dcb-9d39-40e3c7740c6f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:09.881801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 898/1800] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:09 2022] PUT /identity/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a/users/93c728bf1bb9407bbd612e3e0138f43c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:09.888073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:09.893030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eed00-b2e4-43bd-baa8-49163269c198 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.893337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eed00-b2e4-43bd-baa8-49163269c198 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.893633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eed00-b2e4-43bd-baa8-49163269c198 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.905098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 899/1801] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:04:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:04:09.912708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b7938-ca91-483a-8cde-be1bfb6119ca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:09.924213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272458-9ed3-4784-970f-556a9afc93ef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:09.927993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272458-9ed3-4784-970f-556a9afc93ef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:09.927993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272458-9ed3-4784-970f-556a9afc93ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:09.978023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6c272458-9ed3-4784-970f-556a9afc93ef None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:09.979620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c272458-9ed3-4784-970f-556a9afc93ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:10.115679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.116492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.119559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.119986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.121939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.123687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:10.150470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 903/1802] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:10.159343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:10.181284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4976-f31d-460e-8390-d5de3dadf151 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:10.198172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.199689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.200184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.200632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] PATH_INFO: `/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a/users/93c728bf1bb9407bbd612e3e0138f43c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.219861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e38a869e4d42d29a78aa3f93af7f4a, user_id=93c728bf1bb9407bbd612e3e0138f43c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.220136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 900/1803] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:04:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:10.220245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:10.232778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 904/1804] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:10 2022] PUT /identity/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a/users/93c728bf1bb9407bbd612e3e0138f43c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:10.243335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.243673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.243954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.269112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:10.271852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:10.280185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] MFA Rules not processed for user `93c728bf1bb9407bbd612e3e0138f43c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:10.281770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:10.453637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc777c-944e-48cb-8f6b-0dfd9138636f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:10.462830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.466275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.467696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.469381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.491369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.496111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:10.501385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 901/1805] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:10.515345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:10.523140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:10.544308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 905/1806] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:10.613519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:10.616489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:10.672342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d7df1-bc6e-460a-8b24-9700ce401c51 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:10.680106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.680751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.681038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.681310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.704403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.705866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:10.724826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 902/1807] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:10.811386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:10.823310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18600d8-fd58-4958-90bb-53d978c08fee service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:10.829911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b95cbe-5c9a-446e-b864-ce59e311c800 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:10.855320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.855993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.856282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.856566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] PATH_INFO: `/v3/users/edf0eb65a3fa4f189d012d944c8096eb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.857862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf0eb65a3fa4f189d012d944c8096eb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.859930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:10.861307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:10.863053 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:10.865122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:10.865710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] PATH_INFO: `/v3/projects/2e778d9bdec04c009c2e7b3d12514538` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:10.868562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e778d9bdec04c009c2e7b3d12514538)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:10.882815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:11.140766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf0eb65a3fa4f189d012d944c8096eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.142837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf0eb65a3fa4f189d012d944c8096eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.142953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e778d9bdec04c009c2e7b3d12514538', 'request_id': 'req-748b1f0a-a111-41aa-9bf1-c714149d45c3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.173102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf0eb65a3fa4f189d012d944c8096eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.185661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e778d9bdec04c009c2e7b3d12514538', 'request_id': 'req-748b1f0a-a111-41aa-9bf1-c714149d45c3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.195862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf0eb65a3fa4f189d012d944c8096eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.312276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2501aae-4a77-4117-a18a-bd260b6e29a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf0eb65a3fa4f189d012d944c8096eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.324075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 903/1808] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:10 2022] DELETE /identity/v3/users/edf0eb65a3fa4f189d012d944c8096eb => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:11.330940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:11.331227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:11.332433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:11.382019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:11.386640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:11.442685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e778d9bdec04c009c2e7b3d12514538', 'request_id': 'req-748b1f0a-a111-41aa-9bf1-c714149d45c3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.460613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] The token cache is being invalidate because project 2e778d9bdec04c009c2e7b3d12514538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:11.460990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.468364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-748b1f0a-a111-41aa-9bf1-c714149d45c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.472754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 906/1809] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:10 2022] DELETE /identity/v3/projects/2e778d9bdec04c009c2e7b3d12514538 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:11.477914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:11.478291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:11.478618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:11.533214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:11.537205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 904/1810] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:11.537205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:11.543908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:11.547987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:11.562479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 907/1811] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:11.564286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:11.565884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:11.584781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb021a5-4594-41fe-b137-d83901ca8d59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:11.604559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a184b-72a5-4baf-9035-591e24626e39 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:11.608951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:11.614710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:11.615617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:11.616069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:11.616639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:11.617003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:11.617129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] PATH_INFO: `/v3/projects/ada4dc9f9ffe430cba190934fe06f0c6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:11.617726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:11.620745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ada4dc9f9ffe430cba190934fe06f0c6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:11.620903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] PATH_INFO: `/v3/users/85498677028f454291e1f2ed687e2c4b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:11.622271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85498677028f454291e1f2ed687e2c4b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:11.625723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:11.645457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:11.812910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:11.889208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ada4dc9f9ffe430cba190934fe06f0c6', 'request_id': 'req-8360db74-a5d5-483e-ac50-2195fe9e557e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.899116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ada4dc9f9ffe430cba190934fe06f0c6', 'request_id': 'req-8360db74-a5d5-483e-ac50-2195fe9e557e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.917072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85498677028f454291e1f2ed687e2c4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.919186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85498677028f454291e1f2ed687e2c4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:11.947597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85498677028f454291e1f2ed687e2c4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.029162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ada4dc9f9ffe430cba190934fe06f0c6', 'request_id': 'req-8360db74-a5d5-483e-ac50-2195fe9e557e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.044779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] The token cache is being invalidate because project ada4dc9f9ffe430cba190934fe06f0c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:12.045383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.049523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.052484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 905/1812] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:11 2022] DELETE /identity/v3/projects/ada4dc9f9ffe430cba190934fe06f0c6 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:12.053556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.056726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.123857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85498677028f454291e1f2ed687e2c4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.135804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-092c3359-4c2e-481e-84ea-43e3a69f6526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85498677028f454291e1f2ed687e2c4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.158771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 908/1813] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:11 2022] DELETE /identity/v3/users/85498677028f454291e1f2ed687e2c4b => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:12.168049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5dc9-3a3c-4a62-9202-8cdd45fc6f0b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.168049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5dc9-3a3c-4a62-9202-8cdd45fc6f0b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.168049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d5dc9-3a3c-4a62-9202-8cdd45fc6f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.208162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-830d5dc9-3a3c-4a62-9202-8cdd45fc6f0b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:12.213341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-830d5dc9-3a3c-4a62-9202-8cdd45fc6f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.219200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360db74-a5d5-483e-ac50-2195fe9e557e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:12.301961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:12.303120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.303467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.311030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.312038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.347068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 909/1814] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:12.353235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.353235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.353235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.354000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 910/1815] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:04:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:04:12.356579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.358975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.379880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4c850-333a-4526-9d9e-341ba22c06ce None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:12.388446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:12.389096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.393498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.393498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] PATH_INFO: `/v3/users/c24a61479d75455c9fec63658f9a017e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.393498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c24a61479d75455c9fec63658f9a017e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:12.393498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:12.435683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:12.437314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e1d17-85a9-44e7-986b-0815b69852b4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:12.454002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 906/1816] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:12.459299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.459299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.459299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.487614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:12.489068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.504194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 907/1817] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:12.505598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.529415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016e2e-a37d-40ee-97d7-570f71399ffb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:12.539682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:12.540623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.541235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.541663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.543192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:12.543730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:12.545937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ce762-8628-40e5-b7db-6ab50b1dddf7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:12.564344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 908/1818] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:12.566027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.566663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.567101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.569564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:12.631119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:12.632457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.673322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c24a61479d75455c9fec63658f9a017e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.678263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c24a61479d75455c9fec63658f9a017e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.695894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c24a61479d75455c9fec63658f9a017e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.795274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 909/1819] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:12.796251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.797971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.813801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c24a61479d75455c9fec63658f9a017e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.822915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddcee4-856d-4d41-bbfa-2fc1d3e0fb6b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:12.823576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c24a61479d75455c9fec63658f9a017e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:12.838316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:12.839660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.840080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.840470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.841207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:12.843100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 911/1820] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:12 2022] DELETE /identity/v3/users/c24a61479d75455c9fec63658f9a017e => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:12.843583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:12.847912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.852789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:12.931635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff6f8b-d5be-4863-a060-9129c31c159e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:12.948813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 910/1821] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:12 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:12.951083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:12.952992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:12.953942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:12.954781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:12.958127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:12.963920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:13.108549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe26da-d5e6-40ff-b2da-5f618fb9faa5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:13.119796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.123936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.124780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:13.130717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.130717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.130717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.130717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.130717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.151682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 912/1822] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:13.151934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:13.251544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 911/1823] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:12 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:13.259799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:13.288003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e8f9-f774-4cfb-9d46-5a4cec36ab43 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:13.306963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:13.308124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.308679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.309480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.310917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.314339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.336456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 913/1824] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:04:13.349556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:13.366277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7133ff-4526-4bb9-a479-db82a553d20b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:13.373279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:13.376579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.376579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.376579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.376579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.378917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.420191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 912/1825] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:13.435960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:13.454144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39d11-082a-43af-be3e-be4a08c495ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:13.465912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:13.469831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:13.503541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:13.504324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.504610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.504889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] PATH_INFO: `/v3/projects/7cfa33bf2d6a4186af9ef8feaa988442/users/399cccfe5809408e87bae4126f18802b/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.573174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cfa33bf2d6a4186af9ef8feaa988442, user_id=399cccfe5809408e87bae4126f18802b, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.576558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.617755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86a949-d53c-435e-a587-0dc4839f874a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:13.624958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:13.628803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.628803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.628803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.640399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:13.643030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b9077-eebb-4771-81d1-318c914eb195 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:13.659669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 913/1826] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:13.695844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 914/1827] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:13 2022] PUT /identity/v3/projects/7cfa33bf2d6a4186af9ef8feaa988442/users/399cccfe5809408e87bae4126f18802b/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:13.701120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:13.701410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:13.701683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:13.746225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] MFA Rules not processed for user `399cccfe5809408e87bae4126f18802b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:13.749199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:13.942912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 914/1828] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:13.943222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.053084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2e66e-84e6-4a49-b21f-be47e22df7d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:14.060334 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:14.062260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:14.062877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:14.063861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:14.065109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:14.067966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:14.220793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:14.237851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 915/1829] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:13 2022] POST /identity/v3/projects => generated 383 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:04:14.242061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.257081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54e6e8-62cd-460e-baa3-efc2bcd955c1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:14.444105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:14.445689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:14.446072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:14.446444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:14.447976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:14.450365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:14.574430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.583062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 915/1830] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:14 2022] POST /identity/v3/users => generated 298 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:14.609713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ace8-f4c3-4d9e-a6cc-0492bc8c9da8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:14.630549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:14.630549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:14.630549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:14.630549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:14.630820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:14.630820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:14.654066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 916/1831] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:14.658514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.680776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1ce0c-3fc0-4de0-886f-58e160e6fafb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:14.695243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:14.696269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:14.696752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:14.697219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] PATH_INFO: `/v3/projects/80c0d655f00e48aa869caccb2675d656/users/2bec28fe1fed4e4c8e37671d50266281/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:14.746755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c0d655f00e48aa869caccb2675d656, user_id=2bec28fe1fed4e4c8e37671d50266281, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:14.750332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:14.807154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.809004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:14.838732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 916/1832] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:14 2022] PUT /identity/v3/projects/80c0d655f00e48aa869caccb2675d656/users/2bec28fe1fed4e4c8e37671d50266281/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:14.843837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:14.888822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c8e7-aa2a-41e9-bafb-f3b90aec0162 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.103409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.104354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.106354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.106963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] PATH_INFO: `/v3/projects/c6305df0f06d4f2b96ac02bb7a7ebaf8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.110764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6305df0f06d4f2b96ac02bb7a7ebaf8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.116371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.118421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926c662-181b-4d2d-b829-e74e3277b412 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.134376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.135539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.136216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.146094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.150114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.158092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.195631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 917/1833] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:15.196865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:15.221273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99c479-7f13-41f9-acb2-3510b79c412e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.235302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.236033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.236363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.236716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] PATH_INFO: `/v3/projects/80c0d655f00e48aa869caccb2675d656/users/2bec28fe1fed4e4c8e37671d50266281/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.239708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c0d655f00e48aa869caccb2675d656, user_id=2bec28fe1fed4e4c8e37671d50266281, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.242804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74b5a7-c08b-4854-a78f-7d8418e0b657 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.255881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 918/1834] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:15 2022] PUT /identity/v3/projects/80c0d655f00e48aa869caccb2675d656/users/2bec28fe1fed4e4c8e37671d50266281/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:15.268156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.269795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.270233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.313080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] MFA Rules not processed for user `2bec28fe1fed4e4c8e37671d50266281`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:15.323322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:15.348428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6305df0f06d4f2b96ac02bb7a7ebaf8', 'request_id': 'req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.361537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6305df0f06d4f2b96ac02bb7a7ebaf8', 'request_id': 'req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.451363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 919/1835] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:15.467864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:15.469981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:15.477388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6305df0f06d4f2b96ac02bb7a7ebaf8', 'request_id': 'req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.498414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] The token cache is being invalidate because project c6305df0f06d4f2b96ac02bb7a7ebaf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:15.498817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.499581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.501314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 917/1836] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:14 2022] DELETE /identity/v3/projects/c6305df0f06d4f2b96ac02bb7a7ebaf8 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:15.505037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:15.506458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:15.551402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:15.572194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b789a-ba6d-4c3a-a807-13bb0df045e6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.581240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.582688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.583501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.583979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.596300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:15.621734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.626352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.640839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf8095-6ee0-471b-9dc4-a207f4f3aa9f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.647982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.648867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.649398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.649810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] PATH_INFO: `/v3/users/7dd71feab7b2458d8fb7e46f0374c5ef` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.651264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd71feab7b2458d8fb7e46f0374c5ef)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.653301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 920/1837] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:15.654465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:15.655412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.658176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:15.716608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc251-9bcf-4632-9b16-bc23961c26a5 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:15.731119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:15.731931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:15.732117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:15.732395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] PATH_INFO: `/v3/projects/29ed54b31724454f9d35f24a9675b6c6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:15.738063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ed54b31724454f9d35f24a9675b6c6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:15.748337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:15.925265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd71feab7b2458d8fb7e46f0374c5ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.925941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd71feab7b2458d8fb7e46f0374c5ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.951084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ed54b31724454f9d35f24a9675b6c6', 'request_id': 'req-7214ff66-9e96-4b7e-bb16-b3886e65893a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.961147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd71feab7b2458d8fb7e46f0374c5ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:15.988671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ed54b31724454f9d35f24a9675b6c6', 'request_id': 'req-7214ff66-9e96-4b7e-bb16-b3886e65893a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.151901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd71feab7b2458d8fb7e46f0374c5ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.166240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd71feab7b2458d8fb7e46f0374c5ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.174938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 918/1838] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:15 2022] DELETE /identity/v3/users/7dd71feab7b2458d8fb7e46f0374c5ef => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:16.176232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:16.178941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.243251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ed54b31724454f9d35f24a9675b6c6', 'request_id': 'req-7214ff66-9e96-4b7e-bb16-b3886e65893a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.264655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] The token cache is being invalidate because project 29ed54b31724454f9d35f24a9675b6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:16.264655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.268094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.274499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 921/1839] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:15 2022] DELETE /identity/v3/projects/29ed54b31724454f9d35f24a9675b6c6 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:16.275396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:16.280334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.346536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70095425-48bd-4f24-a718-347150a28d96 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:16.352921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:16.353731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:16.354150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:16.355358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] PATH_INFO: `/v3/projects/f609719885454622bc58c8e139a96bbe` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:16.356712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f609719885454622bc58c8e139a96bbe)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:16.359033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:16.423316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214ff66-9e96-4b7e-bb16-b3886e65893a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:16.504192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:16.505634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:16.506244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:16.506808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:16.524830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.543743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f609719885454622bc58c8e139a96bbe', 'request_id': 'req-8caba714-2747-4c30-972f-583212f5588d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.555961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f609719885454622bc58c8e139a96bbe', 'request_id': 'req-8caba714-2747-4c30-972f-583212f5588d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.646608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:16.649829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be25cd8-c0a1-422f-8bc4-5032099ea6c9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:16.655608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f609719885454622bc58c8e139a96bbe', 'request_id': 'req-8caba714-2747-4c30-972f-583212f5588d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.671281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] The token cache is being invalidate because project f609719885454622bc58c8e139a96bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:16.671902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.672992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:16.680864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 919/1840] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:16 2022] DELETE /identity/v3/projects/f609719885454622bc58c8e139a96bbe => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:16.680864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:16.681771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 922/1841] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:16.683615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.685835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0808-82c4-4bf3-a35e-3e46afc7018b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:16.686125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0808-82c4-4bf3-a35e-3e46afc7018b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:16.686491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d0808-82c4-4bf3-a35e-3e46afc7018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:16.708680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-988d0808-82c4-4bf3-a35e-3e46afc7018b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:16.709974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-988d0808-82c4-4bf3-a35e-3e46afc7018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.852006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caba714-2747-4c30-972f-583212f5588d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:16.866904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:16.867715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:16.868122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:16.872316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 923/1842] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:16.873462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:16.873777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:16.874420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:16.876332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] PATH_INFO: `/v3/users/93c728bf1bb9407bbd612e3e0138f43c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:16.879979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c728bf1bb9407bbd612e3e0138f43c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:16.881923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:16.894594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:16.945177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:16.952653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:16.974354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 924/1843] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:16.978456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:16.994555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ce253-047a-4fb3-b002-3952459d3579 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:17.007090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:17.010047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:17.010047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:17.010047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:17.010047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:17.011125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:17.013768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:17.039844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 925/1844] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:04:17.041135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:17.064045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bac99-06bc-4935-b203-ec6acd8028c2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:17.075336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:17.076385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:17.076867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:17.077320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:17.077548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:17.078339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:17.080854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:17.280020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 926/1845] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:17 2022] POST /identity/v3/projects => generated 369 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:17.284409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:17.305528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c728bf1bb9407bbd612e3e0138f43c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:17.306560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c728bf1bb9407bbd612e3e0138f43c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:17.334086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c728bf1bb9407bbd612e3e0138f43c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:17.426159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c728bf1bb9407bbd612e3e0138f43c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:17.438543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c728bf1bb9407bbd612e3e0138f43c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:17.446552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 920/1846] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:16 2022] DELETE /identity/v3/users/93c728bf1bb9407bbd612e3e0138f43c => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:17.486841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:17.488465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:17.520516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474d8f2-811a-4bbe-a7f9-729e71ce4e27 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:17.702266 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:17.703226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:17.703714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:17.704242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:17.705147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:17.708338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:17.720678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d1ea3-9bbd-4025-94ed-1072c5b1a349 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:17.732838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:17.733866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:17.734154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:17.734453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] PATH_INFO: `/v3/users/399cccfe5809408e87bae4126f18802b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:17.738243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=399cccfe5809408e87bae4126f18802b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:17.741114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:17.850359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 927/1847] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:17 2022] POST /identity/v3/users => generated 285 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:17.851236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:17.853348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:17.871996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30e964-499f-45e1-b087-c44439ed9820 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:17.879414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:17.881278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:17.881278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:17.881542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] PATH_INFO: `/v3/projects/50bdc9a2b0854adda828bbf019797ac3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:17.882845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50bdc9a2b0854adda828bbf019797ac3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:17.884987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:18.052367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399cccfe5809408e87bae4126f18802b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.053887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399cccfe5809408e87bae4126f18802b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.066498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50bdc9a2b0854adda828bbf019797ac3', 'request_id': 'req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.074738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399cccfe5809408e87bae4126f18802b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.076062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50bdc9a2b0854adda828bbf019797ac3', 'request_id': 'req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.716011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399cccfe5809408e87bae4126f18802b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.726651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399cccfe5809408e87bae4126f18802b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.734892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 921/1848] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:17 2022] DELETE /identity/v3/users/399cccfe5809408e87bae4126f18802b => generated 0 bytes in 1249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:18.736638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:18.738581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:18.796651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50bdc9a2b0854adda828bbf019797ac3', 'request_id': 'req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.802350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6e0e-26e9-49d9-bf4f-add2e49d9e9d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:18.808194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:18.808803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:18.808909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] The token cache is being invalidate because project 50bdc9a2b0854adda828bbf019797ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:18.809103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:18.809527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.809640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:18.810192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:18.810299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:18.811782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 928/1849] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:17 2022] DELETE /identity/v3/projects/50bdc9a2b0854adda828bbf019797ac3 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:18.811899 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043f49b-0886-4332-8402-d0293e983c88 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:18.812637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:18.816829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:18.822765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 922/1850] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:18.824435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc415-340e-42a5-9a2a-1c68d2cc1ec2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:18.824709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc415-340e-42a5-9a2a-1c68d2cc1ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:18.824977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cc415-340e-42a5-9a2a-1c68d2cc1ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:18.843623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c8cc415-340e-42a5-9a2a-1c68d2cc1ec2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:18.845121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c8cc415-340e-42a5-9a2a-1c68d2cc1ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:18.856275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 923/1851] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:18.857773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:18.858167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:18.858541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:18.871424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=8,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:18.884161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:18.885623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:18.894550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 924/1852] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:18.895506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:18.896824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:18.902131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba49b4-b3e7-4965-87a1-3d1e4c7d0f12 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:18.909019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:18.913113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca96e3-436f-4d97-b459-224411b87bfa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:18.913259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:18.913259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:18.913259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] PATH_INFO: `/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:18.913259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e38a869e4d42d29a78aa3f93af7f4a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:18.913661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:18.915113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:18.915886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:18.916325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:18.916724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:18.917447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:18.919080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:18.929718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 925/1853] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:18.930767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:18.933225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.001363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b9f3-2d46-4b98-8b03-24c7574bae79 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.005179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.005827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.006126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.006570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.016553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.025611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98adb7-1391-416e-ad2a-62c4a2d15c66 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.039816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 926/1854] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:19.041196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.041577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.042525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.071579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:19.073270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.085585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 927/1855] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:19.085585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.094641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204013e-bc6f-4851-b0d6-b15a8d8736a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.100167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.100847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.101126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.101385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] PATH_INFO: `/v3/projects/08557eb335eb4429b32cf2613d9a128f/users/c1385e74fb2a4d2aa3faa2a130b17cb8/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.127178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e38a869e4d42d29a78aa3f93af7f4a', 'request_id': 'req-2b97489d-d62f-47b7-97ae-17fe6ad530d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:19.134690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e38a869e4d42d29a78aa3f93af7f4a', 'request_id': 'req-2b97489d-d62f-47b7-97ae-17fe6ad530d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:19.139419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=08557eb335eb4429b32cf2613d9a128f, user_id=c1385e74fb2a4d2aa3faa2a130b17cb8, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.142514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608ff64-e7a6-45b9-aa41-bd19f8cb5cfe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.240486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e38a869e4d42d29a78aa3f93af7f4a', 'request_id': 'req-2b97489d-d62f-47b7-97ae-17fe6ad530d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:19.251847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] The token cache is being invalidate because project 32e38a869e4d42d29a78aa3f93af7f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:19.252143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:19.252815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:19.254466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 929/1856] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:18 2022] DELETE /identity/v3/projects/32e38a869e4d42d29a78aa3f93af7f4a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:19.255226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.256545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.295575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 928/1857] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:19 2022] PUT /identity/v3/projects/08557eb335eb4429b32cf2613d9a128f/users/c1385e74fb2a4d2aa3faa2a130b17cb8/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:19.298253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed224f7-6293-4e98-bd18-b150aca43c20 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.298253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed224f7-6293-4e98-bd18-b150aca43c20 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.298253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed224f7-6293-4e98-bd18-b150aca43c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.318762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bed224f7-6293-4e98-bd18-b150aca43c20 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:19.320059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bed224f7-6293-4e98-bd18-b150aca43c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.327381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97489d-d62f-47b7-97ae-17fe6ad530d1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.380832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:19.388861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 929/1858] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:19.390189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.390576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.391447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.397334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.398204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.398667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.398941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.399597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:19.400031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.401915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025a5fd-4a59-4c34-8630-0eca29f2c484 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.420637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 930/1859] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:19.420637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.420637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.420637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.425085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] MFA Rules not processed for user `c1385e74fb2a4d2aa3faa2a130b17cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:19.428479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.440830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:19.442042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.453602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 931/1860] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:19.454500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.464458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bd4a4-d2fb-4921-9013-52e37fe207c8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.472468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.473412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.473694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.473972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.474773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.477522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.513611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 930/1861] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:19.514744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.526211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58fcc3-9184-464e-938c-2e78aa549b65 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.530922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.531526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.532329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.532329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.532761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:19.533067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.534656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.546928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 931/1862] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:19.548840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.550959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.565735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913ae10-aba1-454a-8b13-e8bdb78aa639 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.572212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.572212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.572212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.572212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.572636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.574129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.607356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 932/1863] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:19.607356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.668613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 932/1864] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:19.668798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.678979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7c5d6-addd-428b-90bb-018245644a9e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.741407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.741598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e7845-1284-441a-9fad-9f5a13d8948d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.742578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.742958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.743331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.744140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.748866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.748992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.748992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.748992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.748992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.749368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.750340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.865765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 933/1865] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:19.866582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.868644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:19.880477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e855d-5eb4-4ac1-bbf4-33ce8e84736a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.885014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.885654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.885931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.886201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] PATH_INFO: `/v3/projects/7cfa33bf2d6a4186af9ef8feaa988442` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.887296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cfa33bf2d6a4186af9ef8feaa988442)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.889228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:19.904026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 933/1866] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:19.904026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:19.965386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4540-2909-43c1-95c7-cc6d39aa05cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:19.969942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:19.970585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:19.970865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:19.971137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:19.971749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:19.973677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.065344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cfa33bf2d6a4186af9ef8feaa988442', 'request_id': 'req-bd34d640-c268-484f-9b49-fd7b8fb68c9c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.072261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cfa33bf2d6a4186af9ef8feaa988442', 'request_id': 'req-bd34d640-c268-484f-9b49-fd7b8fb68c9c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.169775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 934/1867] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:19 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:20.169775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.177922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5bcfa-7aac-4173-b863-4cdd4300ee73 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.197144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cfa33bf2d6a4186af9ef8feaa988442', 'request_id': 'req-bd34d640-c268-484f-9b49-fd7b8fb68c9c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.212688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] The token cache is being invalidate because project 7cfa33bf2d6a4186af9ef8feaa988442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:20.213083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.214583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.216802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 934/1868] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:19 2022] DELETE /identity/v3/projects/7cfa33bf2d6a4186af9ef8feaa988442 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:20.218070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.219230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.241785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.242473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.242711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.242985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.243690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.245798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.256191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 935/1869] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.257048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.258870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.274993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84d061-c30a-4e95-b0b1-18f146dcdabb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.280757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.281447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.281730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.282094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.282887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.284654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.300405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 936/1870] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.303887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.304669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.314623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34d640-c268-484f-9b49-fd7b8fb68c9c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.319213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.319852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.320104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96828a5-d8ec-4c12-b2aa-84f8ada15ac3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.320207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.320398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.321024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.323580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.328561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.328561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.328561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.328561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] PATH_INFO: `/v3/users/2bec28fe1fed4e4c8e37671d50266281` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.328959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bec28fe1fed4e4c8e37671d50266281)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.330230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.408075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:20.430954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 935/1871] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:20 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:20.431855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.433956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.445795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48683b0-f7e3-425e-8f43-35991610e932 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.450858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.451427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.451706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.452010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.452634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.454209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.464114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 936/1872] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.464961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.474088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dfd67-b687-4993-8d33-68f308664699 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.478548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.481540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.481540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.481540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] PATH_INFO: `/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6/users/0db2ec797113497483018d8b124f032d/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.493622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4a66f21c4e4eef85d92a378df8e6e6, user_id=0db2ec797113497483018d8b124f032d, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.496776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.528269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bec28fe1fed4e4c8e37671d50266281'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.528931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bec28fe1fed4e4c8e37671d50266281'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.535662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bec28fe1fed4e4c8e37671d50266281'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.546966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bec28fe1fed4e4c8e37671d50266281'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.601878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 937/1873] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:20 2022] PUT /identity/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6/users/0db2ec797113497483018d8b124f032d/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:20.602647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.604190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.640639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bec28fe1fed4e4c8e37671d50266281'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:20.647154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 937/1874] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:20 2022] DELETE /identity/v3/users/2bec28fe1fed4e4c8e37671d50266281 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:20.648293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.650829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.679619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f8663-96fc-4ffa-a452-19e29eaf57e3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.752404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:20.756024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4613ca-4708-4c61-a1b1-9c008b842301 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.760867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.761705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.764836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.764836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] PATH_INFO: `/v3/projects/fd9620dbbab4470c884366c70f52b551/users/b658f64f7bd945e489a1cb8f58d8026e/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.768859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.769772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.769772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.770032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.770699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.772430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.778939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9620dbbab4470c884366c70f52b551, user_id=b658f64f7bd945e489a1cb8f58d8026e, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.781711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.782767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 938/1875] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.784067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.785783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.798526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b9e28-ac6b-4c4a-871f-ffa936fbf2c6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.803747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.804372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.804647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.804915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.805546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.807219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.821854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 939/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.821854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.829221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf8457-8ce5-4f93-b524-24a3d208f8c7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.833934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.834792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.836323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.836600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.837242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.838866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.849964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 940/1877] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:20.851081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.861627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5c501-76c2-4399-a848-1a5dfcd9eb43 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:20.866356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:20.867058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.867337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.867630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] PATH_INFO: `/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6/users/0db2ec797113497483018d8b124f032d/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.870353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4a66f21c4e4eef85d92a378df8e6e6, user_id=0db2ec797113497483018d8b124f032d, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:20.873131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddae62c-7fa6-460f-ba95-5d7f90d3a5bc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:20.884245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 938/1878] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:20 2022] PUT /identity/v3/projects/fd9620dbbab4470c884366c70f52b551/users/b658f64f7bd945e489a1cb8f58d8026e/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:20.899092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 941/1879] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:20 2022] PUT /identity/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6/users/0db2ec797113497483018d8b124f032d/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:20.899241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:20.901224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999235-6a97-4f22-86cc-ee3529f237a1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:20.901509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999235-6a97-4f22-86cc-ee3529f237a1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:20.901777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999235-6a97-4f22-86cc-ee3529f237a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:20.931731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-94999235-6a97-4f22-86cc-ee3529f237a1 None None] MFA Rules not processed for user `0db2ec797113497483018d8b124f032d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:20.933533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94999235-6a97-4f22-86cc-ee3529f237a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:20.993298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd887c21-a8bf-4a13-b3db-17f94117daa1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.015824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 942/1880] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:21.015824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.015824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.015824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.024033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:21.048358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] MFA Rules not processed for user `b658f64f7bd945e489a1cb8f58d8026e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:21.049560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.070242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.071083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.071473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.071878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] PATH_INFO: `/v3/projects/69c33868b3584810a25d97570eddee55/users/02a59d6f85844508ada485350b528a57/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.090442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c33868b3584810a25d97570eddee55, user_id=02a59d6f85844508ada485350b528a57, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.093774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.118337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 943/1881] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:21.119212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.121183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.188263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af0e70-5168-47a0-8b97-cedb9e35e660 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.192397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 939/1882] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:20 2022] PUT /identity/v3/projects/69c33868b3584810a25d97570eddee55/users/02a59d6f85844508ada485350b528a57/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:21.192397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.251518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.252716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.252716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.252868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.262211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fe8f2-c2e9-451b-94aa-b08d3ef22b86 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.267823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.267823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.268070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.268170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.268870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.270787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee2b5c-8ec4-4f31-a395-fa778a6a5b07 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.316295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.317812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.328170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 944/1883] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:21.328963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.330331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.344197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259954e-6ce5-4da4-b343-27160741b698 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.349995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.350636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.350916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.351191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] PATH_INFO: `/v3/projects/80c0d655f00e48aa869caccb2675d656` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.352362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c0d655f00e48aa869caccb2675d656)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.354293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.377090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 940/1884] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:21.378600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.378946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.379285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.413918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] MFA Rules not processed for user `02a59d6f85844508ada485350b528a57`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:21.415240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.485714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 941/1885] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:21.486765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.518452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c0d655f00e48aa869caccb2675d656', 'request_id': 'req-b7b921fd-1286-4385-8b41-c333d7a44fca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:21.525296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c0d655f00e48aa869caccb2675d656', 'request_id': 'req-b7b921fd-1286-4385-8b41-c333d7a44fca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:21.557245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a6437-1360-412e-ba78-fdce939cc3e0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.612227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.612859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.613138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.613411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.614053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.615998 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f51959-52d8-4985-a0e8-6cd1142a7761 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.699009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c0d655f00e48aa869caccb2675d656', 'request_id': 'req-b7b921fd-1286-4385-8b41-c333d7a44fca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:21.714841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] The token cache is being invalidate because project 80c0d655f00e48aa869caccb2675d656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:21.715132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:21.715853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:21.717519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 945/1886] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:21 2022] DELETE /identity/v3/projects/80c0d655f00e48aa869caccb2675d656 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:21.718354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.719736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.733058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b921fd-1286-4385-8b41-c333d7a44fca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.739541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 942/1887] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:21.742228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.742228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.742228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.742527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.742527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.742706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.742796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.742879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.743520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.759799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:21.761525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.769469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 943/1888] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:21.770227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.771406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.783142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f1ddb-1ccf-4df0-a125-0000af91956d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.787617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.788220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.788584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.788801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.789435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.790995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232692f-3a48-4959-8213-d89e544240e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.800766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 944/1889] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:21.801905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.802190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.802462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.821355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:21.822527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:21.832020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 945/1890] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:21.832854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.845700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122430ad-d331-4caa-9b78-0f1a0f8ab96b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.851009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.851884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.852352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.852728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] PATH_INFO: `/v3/projects/80b4373f8ace4712b4b822bf15f757ef/users/3b387971b03241e3a289af7be7a71de4/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.872891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b4373f8ace4712b4b822bf15f757ef, user_id=3b387971b03241e3a289af7be7a71de4, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.875593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.878669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 946/1891] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:21.880211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.942805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da06c30-8cb4-41f7-83e4-effdce08118b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.947511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:21.948766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:21.949094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:21.949320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:21.949964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:21.950267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:21.951936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:21.963993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 947/1892] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:21.964796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:21.977232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 946/1893] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:21 2022] PUT /identity/v3/projects/80b4373f8ace4712b4b822bf15f757ef/users/3b387971b03241e3a289af7be7a71de4/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:21.977384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851fea77-31fd-4749-bdde-9d7841c2dc6f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:21.978483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.048458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.049129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.049416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.049687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.050308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.052266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.056858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a320000-9145-4c77-97f6-c36a21f94ae8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.063997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.067887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.067887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.067887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.067887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.067887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.210431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 948/1894] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:22.210431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.214912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4a74a-7eda-43fe-82ed-7e999f69b452 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.219431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.220036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.220315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.220585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.221206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.222771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.233054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 949/1895] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:22.233807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.247412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 947/1896] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:21 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:22.247567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a198e47-eb12-423a-aca0-f0a42da5c6cd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.247822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.305613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.306345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.306625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.306874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.307589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.309665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.313236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fa30-8d5a-45dd-8905-f263156df805 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.318644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.319304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.319603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.319916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] PATH_INFO: `/v3/projects/80b4373f8ace4712b4b822bf15f757ef/users/3b387971b03241e3a289af7be7a71de4/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.323995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 950/1897] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:22.323995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.326057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b4373f8ace4712b4b822bf15f757ef, user_id=3b387971b03241e3a289af7be7a71de4, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.327873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.331353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07780176-139c-4501-87dd-c6ef377c22ce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.336335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.336933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.337212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.337581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.338181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.338485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 948/1898] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:22 2022] PUT /identity/v3/projects/80b4373f8ace4712b4b822bf15f757ef/users/3b387971b03241e3a289af7be7a71de4/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:22.339256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.340325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f28f6-16ea-469c-8ab7-665dbe119264 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.385223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:22.420156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03e5d5-1968-4c81-97d2-46a72bf43550 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.425063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.425715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.426033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.426443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] PATH_INFO: `/v3/projects/b77246441bd441e0ae115ffef1fdc4a2/users/c52fadb10476414c8950326117de0c80/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.442151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77246441bd441e0ae115ffef1fdc4a2, user_id=c52fadb10476414c8950326117de0c80, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.445392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.450517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 951/1899] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:22 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:22.452579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.452890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.453365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.485119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] MFA Rules not processed for user `3b387971b03241e3a289af7be7a71de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:22.486285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:22.539706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 949/1900] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:22 2022] PUT /identity/v3/projects/b77246441bd441e0ae115ffef1fdc4a2/users/c52fadb10476414c8950326117de0c80/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:22.540948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.592702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 952/1901] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:22.595307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.611368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa921-9be7-4f24-b33d-3195f696fd7d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.642854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26083b34-22a7-4234-a96b-75beb63e2537 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.647908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.648582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.649008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.649424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.650151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.653037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.664822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 950/1902] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:22.665722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.669569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:22.707992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:22.719739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.719739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.719739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.719739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.720168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.720168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.734721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 953/1903] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:22.734721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.736527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e0cb-53e3-48c9-912e-22f769023002 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.740319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.740939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.741216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.741491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.747218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7441c9-5d4c-46f4-942f-9988cd39db2b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.753693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.754477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.754993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.754993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.755670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.767140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.770360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 954/1904] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:22.771469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.784517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d45c-ab45-4f28-b41c-6e8f19306d26 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.789239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.790019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.790303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.790571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] PATH_INFO: `/v3/projects/b77246441bd441e0ae115ffef1fdc4a2/users/c52fadb10476414c8950326117de0c80/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.792720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77246441bd441e0ae115ffef1fdc4a2, user_id=c52fadb10476414c8950326117de0c80, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.795563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.810500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 955/1905] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:22 2022] PUT /identity/v3/projects/b77246441bd441e0ae115ffef1fdc4a2/users/c52fadb10476414c8950326117de0c80/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:22.810988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.813115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.831966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b204fca-a95e-43bc-a835-fad5df361537 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.831966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 951/1906] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:22.837121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.837481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.837823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.868772 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] MFA Rules not processed for user `c52fadb10476414c8950326117de0c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:22.870552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:22.879839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97200f0b-decb-482a-969d-94ee2a558d2a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:22.884793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:22.886012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:22.886288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:22.886561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] PATH_INFO: `/v3/projects/2bd3a9a42b3348b8a2b618287c419aed/users/1d0fb24963924a4bb120beb8b236b31f/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:22.900842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd3a9a42b3348b8a2b618287c419aed, user_id=1d0fb24963924a4bb120beb8b236b31f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:22.903510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb4251-e1f5-496f-b95f-c26b6aeb684d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:22.950507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 952/1907] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:22.965034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:22.966658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:23.029480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ac35-a27a-47d5-8f12-33eee3d887cc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:23.031150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:23.031782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:23.032070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:23.032381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:23.042302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:23.046388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:23.048570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 956/1908] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:22 2022] PUT /identity/v3/projects/2bd3a9a42b3348b8a2b618287c419aed/users/1d0fb24963924a4bb120beb8b236b31f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:23.054328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 953/1909] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:23.060175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:23.060460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:23.060738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:23.091127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] MFA Rules not processed for user `1d0fb24963924a4bb120beb8b236b31f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:23.092500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:23.096217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:23.169741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 957/1910] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:23.221301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:23.231742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c87a60-7dbd-4efb-9738-71ac301908cd service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:23.283650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:23.284250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:23.284564 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:23.284883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:23.293595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:23.295085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:23.304308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 954/1911] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:23.354957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:23.367871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ea9a3-910d-49b7-9054-37f79cbb68a1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:23.372494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:23.373230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:23.373954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:23.373954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:23.375844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:23.434431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:23.435911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:23.444815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 958/1912] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:25.756899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:26.011609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:26.083923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fac94-1ef8-4828-afc5-fa94e22a8232 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:26.089472 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c52fadb10476414c8950326117de0c80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b77246441bd441e0ae115ffef1fdc4a2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:26.090108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:26.090393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:26.090850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:26.091503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:26.092054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:26.094592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:26.107184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 955/1913] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:26.111654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:26.124738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9926c35-7392-447e-b150-282f653b74da service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:26.130490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c52fadb10476414c8950326117de0c80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b77246441bd441e0ae115ffef1fdc4a2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:26.131190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:26.131537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:26.131850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:26.132750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:26.138498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:26.211838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:26.279191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 959/1914] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:26 2022] POST /identity/v3/projects => generated 354 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:26.282968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:26.293366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fec0c-ebd5-4d95-a770-5b95408d4a40 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:26.371307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c52fadb10476414c8950326117de0c80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b77246441bd441e0ae115ffef1fdc4a2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:26.373089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:26.373450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:26.374128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:26.374536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:26.374939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:26.376776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:26.396014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 956/1915] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:26.475991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:26.505610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c2bd1-f822-4b59-9e41-436fab3f890b tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:26.515894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c52fadb10476414c8950326117de0c80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b77246441bd441e0ae115ffef1fdc4a2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:26.519860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:26.519860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:26.519860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] PATH_INFO: `/v3/projects/70456138c74740119ce273a1a9afc2c0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:26.521150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorizing `identity:delete_project(project_id=70456138c74740119ce273a1a9afc2c0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:26.523259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:26.680442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70456138c74740119ce273a1a9afc2c0', 'request_id': 'req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:26.689827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70456138c74740119ce273a1a9afc2c0', 'request_id': 'req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:26.765559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70456138c74740119ce273a1a9afc2c0', 'request_id': 'req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:26.786675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] The token cache is being invalidate because project 70456138c74740119ce273a1a9afc2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:26.787137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:26.788371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:26.790694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 960/1916] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:26 2022] DELETE /identity/v3/projects/70456138c74740119ce273a1a9afc2c0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:27.727431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:27.798370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:27.800184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:27.887410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59e1cf-03e0-4efb-ad9b-7505f8be7b2a tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:27.950176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:27.950832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:27.951388 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:27.951671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] PATH_INFO: `/v3/users/02a59d6f85844508ada485350b528a57` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:27.952990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a59d6f85844508ada485350b528a57)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:27.956274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:28.042555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:28.208111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a59d6f85844508ada485350b528a57'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.208958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a59d6f85844508ada485350b528a57'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.218294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a59d6f85844508ada485350b528a57'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.219435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:28.222229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.232626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a59d6f85844508ada485350b528a57'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.239548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea3d5f-97aa-4911-b8d8-9e9e6b9de9c9 tempest-VolumeQuotaClassesTest-1130345672 tempest-VolumeQuotaClassesTest-1130345672-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:28.244928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:28.246060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.246396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.246872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] PATH_INFO: `/v3/users/c1385e74fb2a4d2aa3faa2a130b17cb8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.248610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1385e74fb2a4d2aa3faa2a130b17cb8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:28.250733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:28.316089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-08c156d9-3a54-43e6-9690-deece0673e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a59d6f85844508ada485350b528a57'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.323524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 957/1917] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:27 2022] DELETE /identity/v3/users/02a59d6f85844508ada485350b528a57 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:28.328419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.328700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.329023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.352434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:28.354434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.424786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 958/1918] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:28.434455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:28.436291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.459214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1385e74fb2a4d2aa3faa2a130b17cb8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.461761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1385e74fb2a4d2aa3faa2a130b17cb8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.482832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1385e74fb2a4d2aa3faa2a130b17cb8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.522970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2327c4-9b52-468c-b75b-6dfce21a993f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:28.529023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:28.529627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.529905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.530174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.531913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.544650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:28.546142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0225f8-f47e-4088-895d-15d895e7fc2e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:28.559184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 959/1919] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:28.573550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1385e74fb2a4d2aa3faa2a130b17cb8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.587212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e194fd72-ac83-4104-a7c7-a6c6eaea86fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1385e74fb2a4d2aa3faa2a130b17cb8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:28.604580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.605067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.605364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.609983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 961/1920] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:28 2022] DELETE /identity/v3/users/c1385e74fb2a4d2aa3faa2a130b17cb8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:28.627342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:28.628576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.673393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7a7ad-ff1c-40e5-b0aa-22e886e1513e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.673685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7a7ad-ff1c-40e5-b0aa-22e886e1513e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.673961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7a7ad-ff1c-40e5-b0aa-22e886e1513e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.675438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 962/1921] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:04:28.685460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.685780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.686640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.702364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 960/1922] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:28.708172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:28.709622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.711619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:28.713237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:28.800495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5460e0-bfb4-416d-be5d-321b9313635c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:28.805930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:28.807395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:28.807680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:28.808113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:28.821137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:28.822769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:28.830110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 963/1923] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:28.834077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 961/1924] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:29.039018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:29.041196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:29.068615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af84f9a-63f1-48b6-891a-3d0463086e22 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:29.079821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:29.080984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:29.081758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:29.082227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:29.095917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:29.098339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:29.114554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 964/1925] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:29.284911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:29.286390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:29.316733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf455b9c-bf70-449b-bae0-df24cc66fc53 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:29.325953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:29.327011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:29.327446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:29.328871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] PATH_INFO: `/v3/projects/69c33868b3584810a25d97570eddee55` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:29.335261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c33868b3584810a25d97570eddee55)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:29.343431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:29.519554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c33868b3584810a25d97570eddee55', 'request_id': 'req-aafda201-5923-49a8-8242-69c3367b1820'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.526411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c33868b3584810a25d97570eddee55', 'request_id': 'req-aafda201-5923-49a8-8242-69c3367b1820'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.538280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:29.539743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:29.553466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16de28f-d06d-4192-a5fe-c4fba305f1aa service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:29.592872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c33868b3584810a25d97570eddee55', 'request_id': 'req-aafda201-5923-49a8-8242-69c3367b1820'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.610543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] The token cache is being invalidate because project 69c33868b3584810a25d97570eddee55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:29.611045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.612194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.613767 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 962/1926] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:29 2022] DELETE /identity/v3/projects/69c33868b3584810a25d97570eddee55 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:29.615865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:29.617016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:29.619261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:29.619945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:29.624242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:29.624242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] PATH_INFO: `/v3/users/1d0fb24963924a4bb120beb8b236b31f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:29.624242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0fb24963924a4bb120beb8b236b31f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:29.624242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:29.634702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda201-5923-49a8-8242-69c3367b1820 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:29.639606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:29.640426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:29.640749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:29.641024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] PATH_INFO: `/v3/projects/08557eb335eb4429b32cf2613d9a128f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:29.642171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08557eb335eb4429b32cf2613d9a128f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:29.644122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:29.876293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08557eb335eb4429b32cf2613d9a128f', 'request_id': 'req-ea96a112-476d-4bee-8019-985f50d87980'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.878428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0fb24963924a4bb120beb8b236b31f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.879130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0fb24963924a4bb120beb8b236b31f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.883137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08557eb335eb4429b32cf2613d9a128f', 'request_id': 'req-ea96a112-476d-4bee-8019-985f50d87980'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:29.888834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:29.899530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0fb24963924a4bb120beb8b236b31f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.115448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08557eb335eb4429b32cf2613d9a128f', 'request_id': 'req-ea96a112-476d-4bee-8019-985f50d87980'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.131637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] The token cache is being invalidate because project 08557eb335eb4429b32cf2613d9a128f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:30.131993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.136330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.142650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 963/1927] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:29 2022] DELETE /identity/v3/projects/08557eb335eb4429b32cf2613d9a128f => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:30.142650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:30.142650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:30.211543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96a112-476d-4bee-8019-985f50d87980 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:30.217280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:30.217939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:30.218216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:30.218485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] PATH_INFO: `/v3/users/c52fadb10476414c8950326117de0c80` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:30.219591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52fadb10476414c8950326117de0c80)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:30.221723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:30.257275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0fb24963924a4bb120beb8b236b31f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.265474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0fb24963924a4bb120beb8b236b31f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.271338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 965/1928] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:29 2022] DELETE /identity/v3/users/1d0fb24963924a4bb120beb8b236b31f => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:30.272340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:30.273805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:30.385218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c83a67-fcc5-4477-83af-746358435a1f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:30.391635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:30.392321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:30.392781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:30.393262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] PATH_INFO: `/v3/users/0db2ec797113497483018d8b124f032d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:30.394732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db2ec797113497483018d8b124f032d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:30.396696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:30.435377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52fadb10476414c8950326117de0c80'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.436059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52fadb10476414c8950326117de0c80'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.445387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52fadb10476414c8950326117de0c80'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.458239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52fadb10476414c8950326117de0c80'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.532732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f35ae1ee-4e08-43f3-9ec9-513e1eed6b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52fadb10476414c8950326117de0c80'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.543220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 964/1929] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:30 2022] DELETE /identity/v3/users/c52fadb10476414c8950326117de0c80 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:30.544713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:30.545010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:30.545286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:30.569667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:30.571089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:30.612389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:30.615485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db2ec797113497483018d8b124f032d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.616463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db2ec797113497483018d8b124f032d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.631421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db2ec797113497483018d8b124f032d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.671550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 965/1930] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:30.683356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:30.685383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:30.715582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db2ec797113497483018d8b124f032d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.727857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db2ec797113497483018d8b124f032d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:30.736898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 966/1931] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:30 2022] DELETE /identity/v3/users/0db2ec797113497483018d8b124f032d => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:30.763089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e44b2e-a050-48b0-852f-ca4a5beb5d43 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:30.816377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:30.817068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:30.817353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:30.817801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:30.890161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:30.891603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:30.902656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 966/1932] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:31.156953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:31.158656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:31.175781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dfee5b-26e2-4a3f-b69c-191c8e12f30f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:31.182125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:31.183216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.184224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.184777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] PATH_INFO: `/v3/projects/b77246441bd441e0ae115ffef1fdc4a2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.186910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77246441bd441e0ae115ffef1fdc4a2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:31.190637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:31.359471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77246441bd441e0ae115ffef1fdc4a2', 'request_id': 'req-d118de92-2808-40ec-b51f-249a3786b21d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.366781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77246441bd441e0ae115ffef1fdc4a2', 'request_id': 'req-d118de92-2808-40ec-b51f-249a3786b21d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.437055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:31.440604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:31.488062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77246441bd441e0ae115ffef1fdc4a2', 'request_id': 'req-d118de92-2808-40ec-b51f-249a3786b21d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.507555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08abdd6-c574-40d2-8886-b67c870e6f87 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:31.514151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] The token cache is being invalidate because project b77246441bd441e0ae115ffef1fdc4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:31.515653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.518070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d118de92-2808-40ec-b51f-249a3786b21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.524448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 967/1933] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:31 2022] DELETE /identity/v3/projects/b77246441bd441e0ae115ffef1fdc4a2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:31.530834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7ed3-f5d5-4ac8-9452-3a5045869157 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.531318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7ed3-f5d5-4ac8-9452-3a5045869157 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.531612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd7ed3-f5d5-4ac8-9452-3a5045869157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.590724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b3dd7ed3-f5d5-4ac8-9452-3a5045869157 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:31.593548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b3dd7ed3-f5d5-4ac8-9452-3a5045869157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:31.669666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:31.677704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.678631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.679121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] PATH_INFO: `/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.683037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4a66f21c4e4eef85d92a378df8e6e6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:31.686488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:31.767246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 968/1934] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:31.774879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c333-1155-4cde-978f-863392073730 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.775301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c333-1155-4cde-978f-863392073730 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.775648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43c333-1155-4cde-978f-863392073730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.817844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0f43c333-1155-4cde-978f-863392073730 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:31.819461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f43c333-1155-4cde-978f-863392073730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:31.831445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 969/1935] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:31.837132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43c333-1155-4cde-978f-863392073730 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:31.851925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43c333-1155-4cde-978f-863392073730 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:31.858113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:31.858802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.859136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.859488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.860162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:31.860464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:31.862237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:31.874921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 970/1936] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:04:31.879499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:31.889560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460d502-daf4-447f-b22d-c2b7804c4ff7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:31.899704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:31.900984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:31.901360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:31.901748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:31.902795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:31.904823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:31.914919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4a66f21c4e4eef85d92a378df8e6e6', 'request_id': 'req-ae626397-5404-4f4e-9b9c-38ee07ec2e98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:31.925976 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4a66f21c4e4eef85d92a378df8e6e6', 'request_id': 'req-ae626397-5404-4f4e-9b9c-38ee07ec2e98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.075696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4a66f21c4e4eef85d92a378df8e6e6', 'request_id': 'req-ae626397-5404-4f4e-9b9c-38ee07ec2e98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.083572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 971/1937] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:31 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:04:32.084298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.085955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.091934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] The token cache is being invalidate because project bb4a66f21c4e4eef85d92a378df8e6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:32.092238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.092894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.094724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 967/1938] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:31 2022] DELETE /identity/v3/projects/bb4a66f21c4e4eef85d92a378df8e6e6 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:32.095724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.097309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.159835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863dab-f0b8-4e1b-a1b8-a89677c17fb0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.164620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.165292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.165608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.165885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] PATH_INFO: `/v3/projects/2bd3a9a42b3348b8a2b618287c419aed` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.167227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd3a9a42b3348b8a2b618287c419aed)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.169264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.170810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626397-5404-4f4e-9b9c-38ee07ec2e98 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.175310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.175931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.176210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.176480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.177119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.179000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d796a-556b-4fa0-9f2d-37687bea8a44 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.346777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 968/1939] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:32.346777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.346777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.346777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.359161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd3a9a42b3348b8a2b618287c419aed', 'request_id': 'req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.365764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd3a9a42b3348b8a2b618287c419aed', 'request_id': 'req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.366350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:32.367504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.429190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 969/1940] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:32.436338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.437692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.449421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3fcde-8ee0-4174-99c1-12ebe459359b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.452458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd3a9a42b3348b8a2b618287c419aed', 'request_id': 'req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.461687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.462950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.463243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.463570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] PATH_INFO: `/v3/users/b658f64f7bd945e489a1cb8f58d8026e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.464810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b658f64f7bd945e489a1cb8f58d8026e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.465383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] The token cache is being invalidate because project 2bd3a9a42b3348b8a2b618287c419aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:32.465679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.466341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.466751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.468739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 972/1941] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:32 2022] DELETE /identity/v3/projects/2bd3a9a42b3348b8a2b618287c419aed => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:32.469864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.473658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.487418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a30465-f7f7-4fe7-ab6f-6f02dbd796f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.492669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.493593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.493881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.494147 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.494843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.496538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adff20-b133-4eaf-976e-08901c2d4702 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.507739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 973/1942] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:32.508898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc21f-f6b4-4f58-8f18-be4e96acfaca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.509221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc21f-f6b4-4f58-8f18-be4e96acfaca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.509497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fc21f-f6b4-4f58-8f18-be4e96acfaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.529872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-532fc21f-f6b4-4f58-8f18-be4e96acfaca None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:32.531227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-532fc21f-f6b4-4f58-8f18-be4e96acfaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.542154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 974/1943] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:32.543545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.543959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.544377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.566047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:32.568050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.586204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 975/1944] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:32.586204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.603926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2716e0-84c7-4647-bd37-fd869c18a812 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.612883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.613592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.613934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.614281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.615171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.616924 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.632595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 976/1945] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:32.636797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.646373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03d658-83f7-4766-b7ed-4f17ce42aa1e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.652818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.653531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.653904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.654316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.655203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:32.655607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.657625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c053b29-e272-44f3-ab1e-59c11045ce4f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.671722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 977/1946] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:32.673193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.673478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.673748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.693940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:32.695347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.704245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 978/1947] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:32.708466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.709438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b658f64f7bd945e489a1cb8f58d8026e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.711264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b658f64f7bd945e489a1cb8f58d8026e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.715206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d19b1e-28c2-49f1-9d4e-a631060fa972 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.720035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.720697 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.720974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.721220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b658f64f7bd945e489a1cb8f58d8026e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.721328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] PATH_INFO: `/v3/projects/80867bd7ee7843f8b961928c13550c8e/users/805f2b52efbd4c4eb1fc67b40e4e1641/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.737216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b658f64f7bd945e489a1cb8f58d8026e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.737854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80867bd7ee7843f8b961928c13550c8e, user_id=805f2b52efbd4c4eb1fc67b40e4e1641, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.741748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:32.812085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b658f64f7bd945e489a1cb8f58d8026e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:32.818790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 970/1948] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:32 2022] DELETE /identity/v3/users/b658f64f7bd945e489a1cb8f58d8026e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:32.819835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.821139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.850726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 979/1949] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:32 2022] PUT /identity/v3/projects/80867bd7ee7843f8b961928c13550c8e/users/805f2b52efbd4c4eb1fc67b40e4e1641/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:32.851598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:32.864208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:32.912832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072eca9-e3a8-4411-85b8-850a1c90b602 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:32.988522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:32.989183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:32.989885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:32.990276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:32.991382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:32.993992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.008170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:33.012661 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23688-6ca4-4a50-a808-cf64acc994c0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.017524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.018135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.018411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.018681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.019324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:33.019622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.021248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1012446-c861-45f9-aeed-b77163b14ec1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.038382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 980/1950] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:33.038382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bca0-9ef4-4582-8199-b6536efb205e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.038382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bca0-9ef4-4582-8199-b6536efb205e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.038382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001bca0-9ef4-4582-8199-b6536efb205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.057711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5001bca0-9ef4-4582-8199-b6536efb205e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:33.059178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5001bca0-9ef4-4582-8199-b6536efb205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:33.067715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 981/1951] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:33.068798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.069083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.069359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.100669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 971/1952] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:32 2022] POST /identity/v3/projects => generated 383 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.101354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.104479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] MFA Rules not processed for user `805f2b52efbd4c4eb1fc67b40e4e1641`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:33.106123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:33.181804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525168-2169-4f46-bbb3-b5018e18ffa6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.188377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.188991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.189706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.189994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.190898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.192997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.210458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 982/1953] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:33.213860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.213860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:33.275332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedea0ba-dfe9-479b-b188-40027a65fcd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.279806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.279806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.280017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.280777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.294392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 972/1954] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.295216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.301078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.302549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.313663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 983/1955] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:33.314494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.315989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:33.329605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36164a-4189-4ae1-a910-64ff7264543c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.374507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98786365-645b-4494-ac3d-78ca352d7f8d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.379686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.380318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.380593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.380866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.381492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.383450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.391642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.392282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.392715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.393028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.393812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.395627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.466970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 973/1956] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.467884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.477499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14ba49-c1d5-4d8a-b9e3-bbc6b6183b45 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.482754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.483345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.483625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.483930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.484553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.486494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.503977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:33.514389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 984/1957] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.514389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.575906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 974/1958] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.577106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.579827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cace77-d7d5-4a3d-8098-7c743d04dc8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.584911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.585539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.585833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.586089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.586732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.592181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.602593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 985/1959] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:33.602593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.606607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18d09f-0723-4a6e-889a-1a2f6a3bb556 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.611797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.612543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.613016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.613356 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.614120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.616731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.626541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896dc8c-b3e8-41ba-9249-08cd5a256a69 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.632554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.632992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.633274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.633559 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.634256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.635862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.647269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 986/1960] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:33.648031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.658108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f874fe-72a0-40fe-aa2e-581f49324e4a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.662680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.663279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.663553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.663849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.664483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.666599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.677108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 987/1961] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:33.677947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.689039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef331-68ff-47c5-8988-9a7d661fadfb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.693610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.694212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.694487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.694752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.695381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.697087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.707626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 988/1962] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:33.708890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.719568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9ad42-9d2a-47ee-a115-795fce2a5437 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.722259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 975/1963] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:33.723297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.728447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.729247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.729608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.729982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] PATH_INFO: `/v3/projects/00ca65eb18c549abbf52be9027efabe6/users/eb921431874643959bdcb7601beb2814/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.736617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa1f86-6a5a-4c6e-bea6-f381d73d2cdd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.742877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.744165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.744448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.744719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] PATH_INFO: `/v3/projects/232f67faed57453e99c930bd1fa67279/users/93ab54fa81504882967051ea59092aa4/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.746867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ca65eb18c549abbf52be9027efabe6, user_id=eb921431874643959bdcb7601beb2814, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.749845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275db4b3-2f35-4e43-aa21-81a809d1a8c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.761333 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=232f67faed57453e99c930bd1fa67279, user_id=93ab54fa81504882967051ea59092aa4, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.764007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.864846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 976/1964] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:33 2022] PUT /identity/v3/projects/232f67faed57453e99c930bd1fa67279/users/93ab54fa81504882967051ea59092aa4/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:33.865617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 989/1965] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:33 2022] PUT /identity/v3/projects/00ca65eb18c549abbf52be9027efabe6/users/eb921431874643959bdcb7601beb2814/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:33.866041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:33.886468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.886921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.887320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.922776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] MFA Rules not processed for user `93ab54fa81504882967051ea59092aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:33.925628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:33.948108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f1ff-84b6-4ca8-a284-130d3ac57061 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:33.958374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:33.959021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.959328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.959571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:33.960256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:33.961989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6302ca-2b4d-4825-a8ff-0c22576f6e2a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:33.973954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 977/1966] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:33.975953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:33.977034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:33.977462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.012603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] MFA Rules not processed for user `eb921431874643959bdcb7601beb2814`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:34.014908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:34.054010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 990/1967] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:34.055114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.076388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655fa50-d92c-44dd-bc5a-38cf30551a0d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.083135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.084063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.084355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.084634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] PATH_INFO: `/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f/users/eaa7e7c0a3054b7a844fb07da51689b7/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.105997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba71e7d99f3f4e93b8a710d9359a3f2f, user_id=eaa7e7c0a3054b7a844fb07da51689b7, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.109384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.142368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 978/1968] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:34.143330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.144714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:34.216284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 991/1969] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:34 2022] PUT /identity/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f/users/eaa7e7c0a3054b7a844fb07da51689b7/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:34.217568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.237315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c183b55-3cc8-4c57-a097-373060ce5c6d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.305252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.306429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.306914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.307330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.330027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd392b-c3d0-4a92-9731-f762b0d75155 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.338738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.339615 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.341651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.342708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.344188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.346816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.400597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.402100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.413411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 979/1970] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:34.414625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.440994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b115045-1216-4cce-a20f-443012986221 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.465094 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.465862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.466208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.466545 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.467260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.469968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.477356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 992/1971] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:34 2022] POST /identity/v3/projects => generated 383 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:34.478160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.481208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:34.487380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 980/1972] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:34.487380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.498085 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876dc13-a4f9-401d-9c50-057498a605a8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.612194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dcc9b-6691-43c5-b11d-91bda46b823c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.615380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.616262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.616611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.617045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.617972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.619978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.620703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.620820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.621108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.621452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] PATH_INFO: `/v3/projects/fd9620dbbab4470c884366c70f52b551` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.624829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9620dbbab4470c884366c70f52b551)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.626855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.722018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 981/1973] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:34 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:34.722916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.735375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28581-4da4-43fc-87bd-12c689c0788b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.742193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9620dbbab4470c884366c70f52b551', 'request_id': 'req-f6c9b802-eead-4f06-bc12-f071f0473006'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:34.748702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9620dbbab4470c884366c70f52b551', 'request_id': 'req-f6c9b802-eead-4f06-bc12-f071f0473006'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:34.795235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.795948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.796259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.796539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] PATH_INFO: `/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f/users/eaa7e7c0a3054b7a844fb07da51689b7/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.798547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba71e7d99f3f4e93b8a710d9359a3f2f, user_id=eaa7e7c0a3054b7a844fb07da51689b7, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.801404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.821166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 982/1974] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:34 2022] PUT /identity/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f/users/eaa7e7c0a3054b7a844fb07da51689b7/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:34.821166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.855489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9620dbbab4470c884366c70f52b551', 'request_id': 'req-f6c9b802-eead-4f06-bc12-f071f0473006'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:34.870840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:34.881261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] The token cache is being invalidate because project fd9620dbbab4470c884366c70f52b551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:34.881622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:34.883454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f6c9b802-eead-4f06-bc12-f071f0473006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:34.892191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 993/1975] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:34 2022] DELETE /identity/v3/projects/fd9620dbbab4470c884366c70f52b551 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:34.893849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.894140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.894418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.910342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1418a-d94f-4d80-9e0f-83342ba1ed28 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.916373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.917185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.917482 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.917830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.918573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.920335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:34.928345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] MFA Rules not processed for user `eaa7e7c0a3054b7a844fb07da51689b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:34.929580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:34.934365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 983/1976] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:34.935339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:34.937795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:34.963307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5216b-3959-4f13-b01b-7edf92fb9ccb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:34.969663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:34.970298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:34.970573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:34.970847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] PATH_INFO: `/v3/users/3b387971b03241e3a289af7be7a71de4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:34.972016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b387971b03241e3a289af7be7a71de4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:34.973963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.018896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 994/1977] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:35.020018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:35.021398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.036463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4a670-8f46-475e-8c99-3d69c9c54d85 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:35.042062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:35.042715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.042991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.043260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] PATH_INFO: `/v3/projects/0b1a5a314108475f8edb311e74b5a146/users/830b4d31364f4d7182bd0978b714592b/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.058371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1a5a314108475f8edb311e74b5a146, user_id=830b4d31364f4d7182bd0978b714592b, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:35.061316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.150766 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:35.163901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 995/1978] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:35 2022] PUT /identity/v3/projects/0b1a5a314108475f8edb311e74b5a146/users/830b4d31364f4d7182bd0978b714592b/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:35.164739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:35.167608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.234658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb0a-b5a3-4c13-a61b-4dfe57222bf3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:35.239378 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:35.240043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.240325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.240600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.243938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.323479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b387971b03241e3a289af7be7a71de4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:35.324332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b387971b03241e3a289af7be7a71de4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:35.338137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b387971b03241e3a289af7be7a71de4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:35.350091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:35.353263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.362295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b387971b03241e3a289af7be7a71de4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:35.370614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 996/1979] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:35.371718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:35.470965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5b98b-90b2-49b4-bf1e-4fd5c5e7dab2 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:35.474977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b387971b03241e3a289af7be7a71de4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:35.483196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 984/1980] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:34 2022] DELETE /identity/v3/users/3b387971b03241e3a289af7be7a71de4 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:35.487833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:35.494468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.500419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.500944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.501714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:35.503681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b7f65-62a0-4261-bbfe-3702628bef53 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.526927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 997/1981] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:35.535738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:35.537317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.669587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b0fb-c444-4ca8-8eec-242edbbd6aa7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:35.675896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:35.676650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.676989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.677268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] PATH_INFO: `/v3/projects/0b1a5a314108475f8edb311e74b5a146/users/830b4d31364f4d7182bd0978b714592b/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.679996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1a5a314108475f8edb311e74b5a146, user_id=830b4d31364f4d7182bd0978b714592b, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:35.684212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.702926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 985/1982] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:35 2022] PUT /identity/v3/projects/0b1a5a314108475f8edb311e74b5a146/users/830b4d31364f4d7182bd0978b714592b/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:35.713543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.713904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.714252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.756127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] MFA Rules not processed for user `830b4d31364f4d7182bd0978b714592b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:35.757582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.844595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 998/1983] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:35.853246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:35.855024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:35.941641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06470-0454-4021-8978-a5ef1620721b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:35.946871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:35.947662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:35.948428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:35.948853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:35.967794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:35.968979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:35.981355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 986/1984] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:36.159583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:36.178568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259fd3-ce2e-4198-a972-6f9fe1c608df None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:36.186201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:36.186834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:36.187366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:36.187487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:36.193061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:36.304102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:36.306190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6bb6f-4c06-4a69-8c52-5f7b31df0285 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:36.316032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 999/1985] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:36.397427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:36.399131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:36.516694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c580013-2500-4230-89e7-67659ac7f464 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:36.524384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:36.538897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:36.538897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:36.538897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] PATH_INFO: `/v3/projects/80b4373f8ace4712b4b822bf15f757ef` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:36.538897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b4373f8ace4712b4b822bf15f757ef)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:36.538897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:36.669992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:36.744894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b4373f8ace4712b4b822bf15f757ef', 'request_id': 'req-4f720ac8-0a16-47af-a9c3-9dc341605089'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:36.798665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b4373f8ace4712b4b822bf15f757ef', 'request_id': 'req-4f720ac8-0a16-47af-a9c3-9dc341605089'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:36.874280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:36.885592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b4373f8ace4712b4b822bf15f757ef', 'request_id': 'req-4f720ac8-0a16-47af-a9c3-9dc341605089'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:36.922009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] The token cache is being invalidate because project 80b4373f8ace4712b4b822bf15f757ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:36.922325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:36.925233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f720ac8-0a16-47af-a9c3-9dc341605089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:36.927712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 987/1986] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:36 2022] DELETE /identity/v3/projects/80b4373f8ace4712b4b822bf15f757ef => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:36.934221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:36.934680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:36.935874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:36.963050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:36.964572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:37.048101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1000/1987] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:37.048270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.048270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.048270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.083712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:37.085069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:37.103808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 988/1988] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:37.107536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.129808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98468652-b2ba-42b3-88a2-48f02bdcda91 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.137156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.138070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.138549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.138850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.139713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:37.140500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.142467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.180274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1001/1989] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:37.183996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.197723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8f1a-e56b-408b-b339-633635b3fc15 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.211229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.211943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.212222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.212491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.213141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.215021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.342381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 989/1990] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:37 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:37.346417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.357960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3c15-9d78-4f6b-8465-e9feb963f090 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.472663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.473359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.475911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.475911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.475911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.477001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.574767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1002/1991] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:37 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:37.579647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.592460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d18b6-4aac-4ecb-8674-ef50a96ee4e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.598218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.599256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.599783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.600207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.601093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.602971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.623033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.623664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 990/1992] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:37.634599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f92a-b950-4537-a501-5b92c36e4599 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.640082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.642351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.642351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.642351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.642351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.643622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5cd01-05f0-45c5-b8aa-5b01d30b1eb2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.657664 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1003/1993] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:04:37.662637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:37.677005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaefa-c008-45d2-a8de-eb4947848df9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:37.686971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:37.687645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.687966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.688233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] PATH_INFO: `/v3/projects/25bfe7067f6a430a9a4339bd34428b6a/users/99c2ae0279cf44e2a1ea9eae967245d7/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.706476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25bfe7067f6a430a9a4339bd34428b6a, user_id=99c2ae0279cf44e2a1ea9eae967245d7, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:37.709350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:37.909674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 991/1994] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:37 2022] PUT /identity/v3/projects/25bfe7067f6a430a9a4339bd34428b6a/users/99c2ae0279cf44e2a1ea9eae967245d7/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:37.948367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:37.948367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:37.948367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:37.981961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] MFA Rules not processed for user `99c2ae0279cf44e2a1ea9eae967245d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:37.983352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:38.115848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1004/1995] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:38.155706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:38.161009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:38.226661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca07b8-7fca-4618-af33-59258478d9f1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:38.233184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:38.234128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:38.234569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:38.234845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:38.244895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:38.246233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb82d30-a576-44b7-89f0-261dc7ef327c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:38.256877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 992/1996] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:38.279568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:38.408162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504fcaf-0db9-4145-a2af-312cc5ffe8e7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:38.414936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:38.416183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:38.416463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:38.420953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] PATH_INFO: `/v3/users/99c2ae0279cf44e2a1ea9eae967245d7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:38.423562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c2ae0279cf44e2a1ea9eae967245d7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:38.426270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:38.792586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c2ae0279cf44e2a1ea9eae967245d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:38.793817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c2ae0279cf44e2a1ea9eae967245d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:38.807581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c2ae0279cf44e2a1ea9eae967245d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:38.972816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c2ae0279cf44e2a1ea9eae967245d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:38.987212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c2ae0279cf44e2a1ea9eae967245d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:38.994703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1005/1997] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:38 2022] DELETE /identity/v3/users/99c2ae0279cf44e2a1ea9eae967245d7 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:38.999244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:38.999752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:38.999875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:39.019774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:39.021468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:39.087790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 993/1998] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:39.097298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:39.098765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:39.196649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812da3d-6f5a-4c8f-a9ff-757a04f98e0c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:39.205146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:39.205923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:39.206387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:39.206885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:39.220426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:39.221909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7068b4-4013-477a-bee2-a74287cd011c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:39.241694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1006/1999] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:39.751046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:39.752594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:39.766236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b380e28-9c02-4789-8443-723c053a0288 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:39.772265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:39.773005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:39.773393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:39.773785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] PATH_INFO: `/v3/projects/25bfe7067f6a430a9a4339bd34428b6a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:39.775028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] RBAC: Authorizing `identity:delete_project(project_id=25bfe7067f6a430a9a4339bd34428b6a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:39.777375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.085329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:40.088867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25bfe7067f6a430a9a4339bd34428b6a', 'request_id': 'req-28fa5695-f0e7-4663-8a62-654e9d47302e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:40.095823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25bfe7067f6a430a9a4339bd34428b6a', 'request_id': 'req-28fa5695-f0e7-4663-8a62-654e9d47302e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:40.206068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25bfe7067f6a430a9a4339bd34428b6a', 'request_id': 'req-28fa5695-f0e7-4663-8a62-654e9d47302e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:40.219083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] The token cache is being invalidate because project 25bfe7067f6a430a9a4339bd34428b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:40.219371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:40.220076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-28fa5695-f0e7-4663-8a62-654e9d47302e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:40.221815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 994/2000] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:39 2022] DELETE /identity/v3/projects/25bfe7067f6a430a9a4339bd34428b6a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:40.228115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.228606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.228695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.252556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:40.253669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:40.315872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1007/2001] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:40.320545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:40.325145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.325436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.325740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.345585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:40.346781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:40.354841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 995/2002] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:40.359166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.368966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3be0ea-b252-442c-a293-404980f5003c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.373573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.374173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.374450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.374718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.375355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:40.375654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.377321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.395138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1008/2003] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:40.396286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.406586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2577f9-724a-4bb4-8ca5-b4ee747a617a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.411678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.412322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.412606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.412877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.413569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.415831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.562937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 996/2004] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:40 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:04:40.567037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.578163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cf18e-a591-40d0-80b2-85de782d8a20 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.636265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.636867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.637151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.637408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.638018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.639954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.757417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1009/2005] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:40 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:40.762007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.774498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766f9e6-58bf-42a0-b1d7-ef0a6544939c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.779454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.780148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.780498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.780847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.781593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.783196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.793656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 997/2006] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:40.802173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.813051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273379a2-04d9-4720-a04a-372a6d10a9dd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.818109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.818895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.819313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.819719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.820557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.822886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e094fb-1c16-493f-ae5a-f02cd30e7401 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:40.837899 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1010/2007] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:40.842081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:40.854099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f872-478c-4295-a8d1-38b84e056aa1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:40.860389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:40.861178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:40.861555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:40.861974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] PATH_INFO: `/v3/projects/2b7f7b3653564f87b650fc826da1dfde/users/626cb056cd074662ab13450c49aa9e0f/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:40.881189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7f7b3653564f87b650fc826da1dfde, user_id=626cb056cd074662ab13450c49aa9e0f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:40.884038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:41.083713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 998/2008] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:40 2022] PUT /identity/v3/projects/2b7f7b3653564f87b650fc826da1dfde/users/626cb056cd074662ab13450c49aa9e0f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:41.099354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:41.099634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:41.099935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:41.131183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] MFA Rules not processed for user `626cb056cd074662ab13450c49aa9e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:41.132486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:41.201435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1011/2009] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:41.212908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:41.215572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:41.281591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d4cef-69a8-4067-9cd5-03f4253d5c44 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:41.285278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:41.285880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:41.286162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:41.286436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:41.295421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:41.296902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9a19-ea6e-4fb8-a599-e6671585851d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:41.305975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 999/2010] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:41.384829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:41.386308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:41.457560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8b46a-007f-4c64-8582-db6ca4ee8ad1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:41.463921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:41.464821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:41.465152 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:41.465430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] PATH_INFO: `/v3/users/eb921431874643959bdcb7601beb2814` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:41.466644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb921431874643959bdcb7601beb2814)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:41.468639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:41.693106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb921431874643959bdcb7601beb2814'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:41.693756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb921431874643959bdcb7601beb2814'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:41.705639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb921431874643959bdcb7601beb2814'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:41.863296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb921431874643959bdcb7601beb2814'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:41.870599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb921431874643959bdcb7601beb2814'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:41.876470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1012/2011] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:41 2022] DELETE /identity/v3/users/eb921431874643959bdcb7601beb2814 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:41.881458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:41.881794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:41.882070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:41.905021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:41.906669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:41.974208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1000/2012] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:41.980285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:41.981851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:42.045380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a4b79-687d-45b2-8098-de2cc32c6d06 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:42.050030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:42.050673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:42.050953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:42.051226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:42.061694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:42.063521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:42.073485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1013/2013] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:42.218798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:42.220538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:42.282172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a03ac-31d0-4fa9-a550-a53977cc1805 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:42.286790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:42.287667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:42.287983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:42.288373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:42.290295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:42.396347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:42.397799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:42.409909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1001/2014] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:42.567627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:42.569493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:42.586110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39d78-08ea-4c42-9611-7b631665f739 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:42.590958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:42.600488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:42.600488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:42.600488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] PATH_INFO: `/v3/projects/00ca65eb18c549abbf52be9027efabe6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:42.600488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ca65eb18c549abbf52be9027efabe6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:42.600488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:42.715104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ca65eb18c549abbf52be9027efabe6', 'request_id': 'req-745880fb-f064-462d-a533-5040689594d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:42.721398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ca65eb18c549abbf52be9027efabe6', 'request_id': 'req-745880fb-f064-462d-a533-5040689594d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:42.827884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ca65eb18c549abbf52be9027efabe6', 'request_id': 'req-745880fb-f064-462d-a533-5040689594d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:42.838526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] The token cache is being invalidate because project 00ca65eb18c549abbf52be9027efabe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:42.838823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:42.839464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:42.840970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1014/2015] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:42 2022] DELETE /identity/v3/projects/00ca65eb18c549abbf52be9027efabe6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:42.845109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:42.855195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a34ca-f725-40ba-9ae3-9f5d41ddb142 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:42.909783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:42.910407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:42.910689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:42.910962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] PATH_INFO: `/v3/users/830b4d31364f4d7182bd0978b714592b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:42.912103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830b4d31364f4d7182bd0978b714592b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:42.914016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:43.216220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830b4d31364f4d7182bd0978b714592b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:43.216918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830b4d31364f4d7182bd0978b714592b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:43.225504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830b4d31364f4d7182bd0978b714592b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:43.237856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:43.247971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830b4d31364f4d7182bd0978b714592b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:43.377950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c9111d97-e905-4ae2-87e9-11c47fde2501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830b4d31364f4d7182bd0978b714592b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:43.385999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1002/2016] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:42 2022] DELETE /identity/v3/users/830b4d31364f4d7182bd0978b714592b => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:43.589034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:43.862771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:43.864322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:43.879628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745880fb-f064-462d-a533-5040689594d8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:43.958011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:43.958649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:43.958924 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:43.959195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] PATH_INFO: `/v3/projects/0b1a5a314108475f8edb311e74b5a146` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:43.960403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1a5a314108475f8edb311e74b5a146)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:43.962914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.161776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1a5a314108475f8edb311e74b5a146', 'request_id': 'req-63d34beb-9632-46dd-832a-a70828da23e5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:44.168597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1a5a314108475f8edb311e74b5a146', 'request_id': 'req-63d34beb-9632-46dd-832a-a70828da23e5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:44.267989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1a5a314108475f8edb311e74b5a146', 'request_id': 'req-63d34beb-9632-46dd-832a-a70828da23e5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:44.279667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] The token cache is being invalidate because project 0b1a5a314108475f8edb311e74b5a146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:44.279990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:44.280908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63d34beb-9632-46dd-832a-a70828da23e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:44.282654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1015/2017] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:43 2022] DELETE /identity/v3/projects/0b1a5a314108475f8edb311e74b5a146 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:44.287668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.288099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.288282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.310288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:44.311634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:44.380194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1003/2018] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:44.406250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.406742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.407029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.430995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:44.432458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:44.441978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1016/2019] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:44.446893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.457624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac03ea-a189-465c-b0b3-3f8631f248c3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.465636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:44.466093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.466510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.466791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.467451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:44.468009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:44.470221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.490229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1004/2020] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:04:44.497023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.513858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d00b3-7862-49af-b0a6-930134ca8d10 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.519297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:44.520023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.520590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.520815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.521542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:44.523549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.636088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1017/2021] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:44 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:44.640424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.650863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d9f3e-5948-4268-a22a-65ca64b5538d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.710816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:44.712090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.712373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.712650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.713275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:44.715178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.903847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1005/2022] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:44 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:44.907990 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.917997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8d5e-1b6b-47ba-b216-9a5a6d037338 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.922598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:44.923173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.923499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.923714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.924373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:44.926066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.936863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1018/2023] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:44.942135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.953171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efc200-6c25-4d16-aa74-6fa842ec1609 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.959068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:44.959675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:44.959989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:44.960259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:44.967853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:44.967853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e18f76-8154-4dd7-97d9-24c2e15c9ebc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:44.980189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1006/2024] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:44.981292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:44.991234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5996f1-0606-4c38-9d2e-939b22d1b9de admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:44.996253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.003332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.003332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.003332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] PATH_INFO: `/v3/projects/076b43a7547f483593bc8316d9da8191/users/ffa819bab8fa4a1aaf3dcfac771bd36c/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.012495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076b43a7547f483593bc8316d9da8191, user_id=ffa819bab8fa4a1aaf3dcfac771bd36c, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.015179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:45.097306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1019/2025] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:44 2022] PUT /identity/v3/projects/076b43a7547f483593bc8316d9da8191/users/ffa819bab8fa4a1aaf3dcfac771bd36c/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:45.108299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.109004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.109465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.138477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] MFA Rules not processed for user `ffa819bab8fa4a1aaf3dcfac771bd36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:45.139741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:45.206839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1007/2026] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:45.211665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:45.273080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22e375-52cf-4bc1-8255-833d2de00714 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:45.279602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.280324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.280631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.280954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.281616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.283582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:45.446560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1020/2027] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:45 2022] POST /identity/v3/projects => generated 377 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:45.450028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:45.460529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65e70-9b2e-49ef-b0ef-53ed9caee51d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:45.533206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.533903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.534252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.534591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.535491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.537518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:45.694924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1008/2028] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:45 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:45.706745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:45.717316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd81b5-5d9e-4801-bca7-72017700c881 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:45.722178 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.722811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.723091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.723364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.724051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.725655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:45.736656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1021/2029] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:45.740878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:45.751409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e804a8-ed79-48c1-859c-e1f70dd11b7e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:45.756695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.757303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.757583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.757887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.758524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.760176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cad36-7fbd-413b-a266-711e8e72cb1e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:45.770934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1009/2030] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:04:45.775318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:45.785207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0104e-c241-4b0e-a8d5-02fdd13ac3d2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:45.790390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:45.791045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:45.791322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:45.791592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] PATH_INFO: `/v3/projects/bd305e35bafe479083e29e3865d0879c/users/a0b3e09f559e4384ad7f06bf1be5a3a5/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:45.806903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd305e35bafe479083e29e3865d0879c, user_id=a0b3e09f559e4384ad7f06bf1be5a3a5, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:45.809847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:46.012455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1022/2031] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:45 2022] PUT /identity/v3/projects/bd305e35bafe479083e29e3865d0879c/users/a0b3e09f559e4384ad7f06bf1be5a3a5/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:46.023204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.023631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.023872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.052461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] MFA Rules not processed for user `a0b3e09f559e4384ad7f06bf1be5a3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:46.053700 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:46.120433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1010/2032] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:46.125106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:46.186107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2dfb3-e065-496f-aac5-7eafada040f4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:46.190578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:46.191170 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.191443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.191713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.192348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:46.194223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:46.458187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1023/2033] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:46 2022] POST /identity/v3/projects => generated 377 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:46.462235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:46.474411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee7224-5fb4-4052-bae1-ad5a1e3ad6e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:46.535481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:46.536161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.536444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.536751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.537395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:46.540015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:46.663698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:46.669020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1011/2034] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:46 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:46.675840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:46.698283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65f77-1fa1-47c4-956a-c32ee9e0c60e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:46.707884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:46.708576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.708941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.709308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.710054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:46.712291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:46.725253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1024/2035] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:46.736649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:46.741125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a70286-70e6-41b4-b00f-ab97ff0dcb9c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:46.756887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:46.757645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.758341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.758715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] PATH_INFO: `/v3/projects/c744b363c30a4bb4926a1e03b8f4f682/users/93834c4ca6e345fb8e2218a207dcc17e/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.777603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:46.783168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c744b363c30a4bb4926a1e03b8f4f682, user_id=93834c4ca6e345fb8e2218a207dcc17e, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:46.786199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:46.910691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1012/2036] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:46 2022] PUT /identity/v3/projects/c744b363c30a4bb4926a1e03b8f4f682/users/93834c4ca6e345fb8e2218a207dcc17e/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:46.915573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:46.926959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f79fc-1057-48d2-94c2-561cb49adf5f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:46.997089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:46.999832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:46.999832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:46.999832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:46.999832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:47.000753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b728705-70da-4996-92f1-d3d0bd02845f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:47.011636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1025/2037] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:47.016730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:47.029929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0261-1510-4532-98cf-d372bfe5d139 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:47.034769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:47.035620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:47.036001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:47.036348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] PATH_INFO: `/v3/projects/c744b363c30a4bb4926a1e03b8f4f682/users/93834c4ca6e345fb8e2218a207dcc17e/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:47.038355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c744b363c30a4bb4926a1e03b8f4f682, user_id=93834c4ca6e345fb8e2218a207dcc17e, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:47.041197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:47.052579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1013/2038] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:47 2022] PUT /identity/v3/projects/c744b363c30a4bb4926a1e03b8f4f682/users/93834c4ca6e345fb8e2218a207dcc17e/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:47.064504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:47.064859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:47.065133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:47.093954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] MFA Rules not processed for user `93834c4ca6e345fb8e2218a207dcc17e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:47.095291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:47.165866 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1026/2039] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:47.177670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:47.179376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:47.237770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828961fb-312b-40be-b20f-532445f94e5f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:47.241412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:47.242843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:47.243140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:47.243409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:47.252658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:47.254084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:47.263269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1014/2040] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:47.372709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:47.374140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:47.387200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5d65-4806-4525-bd69-43e525bab17b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:47.390711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:47.391301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:47.391574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:47.391880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:47.448093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:47.449527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eefd97-561f-4b2f-bb9a-21323f13b78a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:47.458416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1027/2041] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:48.343105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:48.371466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:49.182725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:49.205458 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:49.657099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:49.673851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:49.891272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.188126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.234086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.519698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.673858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.731091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.891290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:50.942967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.005342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.056956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.091501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.445525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.491520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:51.982787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:52.045576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:52.553724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:52.588778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:52.925684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:53.344346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:53.354281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:53.355925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:53.427126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532249e-ec71-49f9-8beb-1a933377cd59 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:53.433205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:53.433818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:53.434175 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:53.435885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] PATH_INFO: `/v3/users/805f2b52efbd4c4eb1fc67b40e4e1641` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:53.436562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805f2b52efbd4c4eb1fc67b40e4e1641)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:53.438590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:53.616595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:53.698565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805f2b52efbd4c4eb1fc67b40e4e1641'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:53.702107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805f2b52efbd4c4eb1fc67b40e4e1641'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:53.709557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805f2b52efbd4c4eb1fc67b40e4e1641'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:53.713977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:53.721913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805f2b52efbd4c4eb1fc67b40e4e1641'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:53.794456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805f2b52efbd4c4eb1fc67b40e4e1641'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:53.801672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1015/2042] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:53 2022] DELETE /identity/v3/users/805f2b52efbd4c4eb1fc67b40e4e1641 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:53.806758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:53.807115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:53.807479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:53.827032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:53.828384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:53.889034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1028/2043] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:53.899699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:53.901234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:53.984437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966cdf-6b4d-4067-bf93-0daa7f108f09 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:53.989862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:53.990619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:53.990925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:53.992069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:54.004335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:54.005943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:54.016997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1016/2044] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:54.094146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:54.263068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:54.607009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:54.614853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:54.616453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:54.630927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89fa7-8256-47d5-9455-77aebe72148f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:54.636345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:54.641190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:54.641190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:54.641190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] PATH_INFO: `/v3/projects/80867bd7ee7843f8b961928c13550c8e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:54.641190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80867bd7ee7843f8b961928c13550c8e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:54.641190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:54.766689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80867bd7ee7843f8b961928c13550c8e', 'request_id': 'req-8c007ef8-af59-448c-a9b7-a465e6f91019'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:54.774162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80867bd7ee7843f8b961928c13550c8e', 'request_id': 'req-8c007ef8-af59-448c-a9b7-a465e6f91019'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:54.829436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:54.863056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80867bd7ee7843f8b961928c13550c8e', 'request_id': 'req-8c007ef8-af59-448c-a9b7-a465e6f91019'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:54.880459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] The token cache is being invalidate because project 80867bd7ee7843f8b961928c13550c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:54.880945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:54.881605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:54.892104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1029/2045] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:54 2022] DELETE /identity/v3/projects/80867bd7ee7843f8b961928c13550c8e => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:54.897563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:54.908675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570a4c-5bbf-42c3-85d6-19785d5fbae2 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:54.965620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:54.966252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:54.966530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:54.966803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] PATH_INFO: `/v3/users/eaa7e7c0a3054b7a844fb07da51689b7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:54.967946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa7e7c0a3054b7a844fb07da51689b7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:54.969874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:55.141203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:55.190582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa7e7c0a3054b7a844fb07da51689b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.191235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa7e7c0a3054b7a844fb07da51689b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.198560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa7e7c0a3054b7a844fb07da51689b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.210955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa7e7c0a3054b7a844fb07da51689b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.273672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa7e7c0a3054b7a844fb07da51689b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.281594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1017/2046] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:54 2022] DELETE /identity/v3/users/eaa7e7c0a3054b7a844fb07da51689b7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:55.297192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:55.488077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:55.489612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:55.502725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c007ef8-af59-448c-a9b7-a465e6f91019 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:55.529826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:55.559608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:55.560308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:55.560592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:55.560864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] PATH_INFO: `/v3/users/ffa819bab8fa4a1aaf3dcfac771bd36c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:55.562171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa819bab8fa4a1aaf3dcfac771bd36c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:55.564236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:55.628212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:55.836642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:55.837762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:55.845016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa819bab8fa4a1aaf3dcfac771bd36c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.845829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa819bab8fa4a1aaf3dcfac771bd36c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.850920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80161a59-b65e-4bc0-9419-20959a58ea34 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:55.856510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:55.857424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:55.857751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:55.857991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] PATH_INFO: `/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:55.859132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba71e7d99f3f4e93b8a710d9359a3f2f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:55.861241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:55.865071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa819bab8fa4a1aaf3dcfac771bd36c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.979683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa819bab8fa4a1aaf3dcfac771bd36c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:55.994747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f708aa0-71a1-4869-be31-b7a5a0cd0ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa819bab8fa4a1aaf3dcfac771bd36c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.001544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1030/2047] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:55 2022] DELETE /identity/v3/users/ffa819bab8fa4a1aaf3dcfac771bd36c => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:56.007569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.008576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.008880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.018783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba71e7d99f3f4e93b8a710d9359a3f2f', 'request_id': 'req-d2c10bac-5d85-4e5d-834e-4357d1b5473b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.030559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba71e7d99f3f4e93b8a710d9359a3f2f', 'request_id': 'req-d2c10bac-5d85-4e5d-834e-4357d1b5473b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.034950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:56.036747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.108054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1031/2048] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:56.118964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.122077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.159044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba71e7d99f3f4e93b8a710d9359a3f2f', 'request_id': 'req-d2c10bac-5d85-4e5d-834e-4357d1b5473b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.171209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] The token cache is being invalidate because project ba71e7d99f3f4e93b8a710d9359a3f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:56.171781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.172684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d2c10bac-5d85-4e5d-834e-4357d1b5473b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.174191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1018/2049] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:55 2022] DELETE /identity/v3/projects/ba71e7d99f3f4e93b8a710d9359a3f2f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:56.179955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b00ca-3c53-4b81-aac5-2e8d631797af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.180245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b00ca-3c53-4b81-aac5-2e8d631797af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.180520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b00ca-3c53-4b81-aac5-2e8d631797af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.187382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34654e-9636-43c5-babd-1b269f3f2a70 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.192279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.192923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.193320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.193607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.197841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.200893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4e9b00ca-3c53-4b81-aac5-2e8d631797af None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:56.209208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e9b00ca-3c53-4b81-aac5-2e8d631797af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.210575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1019/2050] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:56.210708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.212695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.220539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.220825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.221272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.231968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1032/2051] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:56.254684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:56.256016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.265949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1020/2052] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:56.269905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.280256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f070-308d-44b7-9667-7705088b8592 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.285535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.286135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.286437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.286712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.287469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:04:56.287797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.289457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.302390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1033/2053] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:04:56.307670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.316257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:56.319833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9acce5-752f-4649-ad21-e21a7dc4f954 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.331065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.331811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.332201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.332811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.333533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.337951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.438895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1021/2054] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:04:56.440180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.450818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662e1ff-cbd2-41d7-9189-4d3bbf980e46 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.496588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:56.512427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.513039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.513319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.513592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.514212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.516195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.617742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1034/2055] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:04:56.621890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.632018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a09788-7852-44a3-aae8-86a565a24ba9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.637268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.637862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.638148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.638418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.639056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.640670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.653066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1022/2056] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:56.657038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.667975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0bc4b-fda3-4f1a-8502-240236827bf8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.672707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.673294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.673571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.673841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.674478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.676107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.687324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1035/2057] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:04:56.709909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.711109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.711109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.711277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4ef07-3977-4abe-853d-73c1c4088a58 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.711371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94706bc-3fe5-40b3-b000-3465bcac6d8d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:56.713493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.714186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.714471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.714751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] PATH_INFO: `/v3/projects/c9d5402587864961a3e3842870114d49/users/3645ab56f5104d84b46bf4f89e9815ee/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.715883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:56.716584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.716857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.717127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] PATH_INFO: `/v3/projects/076b43a7547f483593bc8316d9da8191` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.718269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=076b43a7547f483593bc8316d9da8191)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.724815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.735256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d5402587864961a3e3842870114d49, user_id=3645ab56f5104d84b46bf4f89e9815ee, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:56.738042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86f839-d0cb-470e-bf2f-057b1e0fe90c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:56.833547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1023/2058] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:04:56 2022] PUT /identity/v3/projects/c9d5402587864961a3e3842870114d49/users/3645ab56f5104d84b46bf4f89e9815ee/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:56.844579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076b43a7547f483593bc8316d9da8191', 'request_id': 'req-246fdf00-97bd-4934-8380-ad537b4a9dda'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.845316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:56.845794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:56.845894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:56.851005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076b43a7547f483593bc8316d9da8191', 'request_id': 'req-246fdf00-97bd-4934-8380-ad537b4a9dda'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.875211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] MFA Rules not processed for user `3645ab56f5104d84b46bf4f89e9815ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:56.876464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.919108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076b43a7547f483593bc8316d9da8191', 'request_id': 'req-246fdf00-97bd-4934-8380-ad537b4a9dda'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.931137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] The token cache is being invalidate because project 076b43a7547f483593bc8316d9da8191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:56.931427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.932083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:56.933948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1036/2059] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:56 2022] DELETE /identity/v3/projects/076b43a7547f483593bc8316d9da8191 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:56.938702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.940351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:56.942714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1024/2060] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:56.954295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:56.955824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:57.011419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fdf00-97bd-4934-8380-ad537b4a9dda admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:57.013086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc6706-2c94-4c41-8222-5d056b2b693b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:57.016056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:57.016677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:57.016799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:57.016951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:57.017238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] PATH_INFO: `/v3/users/a0b3e09f559e4384ad7f06bf1be5a3a5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:57.017349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:57.017619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:57.017892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:57.018455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b3e09f559e4384ad7f06bf1be5a3a5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:57.019668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:57.020415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:57.030890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:57.032607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:57.042176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1025/2061] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:57.066821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:57.068273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:57.081609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211081-2dda-410a-9a9f-20851781b1c9 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:57.087101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:57.087719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:57.088037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:57.089342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] PATH_INFO: `/v3/users/3645ab56f5104d84b46bf4f89e9815ee` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:57.089599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3645ab56f5104d84b46bf4f89e9815ee)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:57.091611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:57.310801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b3e09f559e4384ad7f06bf1be5a3a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.312166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b3e09f559e4384ad7f06bf1be5a3a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.324363 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b3e09f559e4384ad7f06bf1be5a3a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.356729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3645ab56f5104d84b46bf4f89e9815ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.357413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3645ab56f5104d84b46bf4f89e9815ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.364254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3645ab56f5104d84b46bf4f89e9815ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.376439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3645ab56f5104d84b46bf4f89e9815ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.445628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b3e09f559e4384ad7f06bf1be5a3a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.456190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d227eb3b-4e88-4088-933b-56ee4fa0895e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b3e09f559e4384ad7f06bf1be5a3a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.466600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1037/2062] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:56 2022] DELETE /identity/v3/users/a0b3e09f559e4384ad7f06bf1be5a3a5 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:57.536337 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3645ab56f5104d84b46bf4f89e9815ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:57.547017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1026/2063] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:57 2022] DELETE /identity/v3/users/3645ab56f5104d84b46bf4f89e9815ee => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:04:57.548768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:57.549117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:57.549407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:57.593629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:57.594940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:57.697813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1038/2064] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:04:57.719587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:57.721079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:57.792688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fa4a3-c57d-497d-b44b-e4ecc568f1c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:57.796427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:57.797189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:57.797418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:57.797688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:57.808177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:57.809755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:57.819924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1027/2065] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:04:57.862520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:58.234159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:58.458487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:58.459960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:58.482284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31269a3-2405-4d0a-b33c-62109fc9c9fb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:58.501495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:58.501495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:58.501495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:58.501495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] PATH_INFO: `/v3/projects/bd305e35bafe479083e29e3865d0879c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:58.509608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd305e35bafe479083e29e3865d0879c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:58.510047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:58.511728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:58.512326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:58.540215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f84dc-5930-4404-9eca-e3f77fa9a067 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:58.556469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:58.557644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:58.558282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:58.558703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] PATH_INFO: `/v3/projects/c9d5402587864961a3e3842870114d49` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:58.560329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d5402587864961a3e3842870114d49)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:58.569698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:58.702577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd305e35bafe479083e29e3865d0879c', 'request_id': 'req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.709110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd305e35bafe479083e29e3865d0879c', 'request_id': 'req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.766782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d5402587864961a3e3842870114d49', 'request_id': 'req-4084607f-fe6d-46b3-814f-891a61eb74be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.774508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d5402587864961a3e3842870114d49', 'request_id': 'req-4084607f-fe6d-46b3-814f-891a61eb74be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.827279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd305e35bafe479083e29e3865d0879c', 'request_id': 'req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.838930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] The token cache is being invalidate because project bd305e35bafe479083e29e3865d0879c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:58.839156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.839812 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.841303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1039/2066] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:58 2022] DELETE /identity/v3/projects/bd305e35bafe479083e29e3865d0879c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:58.842061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:58.843329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:58.906425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a7d2f-8e7e-4fb5-8498-1f18b0456fe6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:58.919278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d5402587864961a3e3842870114d49', 'request_id': 'req-4084607f-fe6d-46b3-814f-891a61eb74be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.930975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] The token cache is being invalidate because project c9d5402587864961a3e3842870114d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:04:58.931298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.931982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:58.933795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1028/2067] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:58 2022] DELETE /identity/v3/projects/c9d5402587864961a3e3842870114d49 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:58.934510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:58.936002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:58.980273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:58.981201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:58.981483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:58.981761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] PATH_INFO: `/v3/users/626cb056cd074662ab13450c49aa9e0f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:58.983075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626cb056cd074662ab13450c49aa9e0f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:58.985105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:59.015942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084607f-fe6d-46b3-814f-891a61eb74be admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:59.020760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:59.021418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:59.021705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:59.021981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] PATH_INFO: `/v3/users/93834c4ca6e345fb8e2218a207dcc17e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:59.023240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93834c4ca6e345fb8e2218a207dcc17e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:59.025241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:59.230048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626cb056cd074662ab13450c49aa9e0f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.231514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626cb056cd074662ab13450c49aa9e0f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.243203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626cb056cd074662ab13450c49aa9e0f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.324680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626cb056cd074662ab13450c49aa9e0f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.332510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099fb61c-2d6b-47b4-ba14-1acbcfc95993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626cb056cd074662ab13450c49aa9e0f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.340264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93834c4ca6e345fb8e2218a207dcc17e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.341478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93834c4ca6e345fb8e2218a207dcc17e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.348728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93834c4ca6e345fb8e2218a207dcc17e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.359276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1040/2068] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:58 2022] DELETE /identity/v3/users/626cb056cd074662ab13450c49aa9e0f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:59.361921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93834c4ca6e345fb8e2218a207dcc17e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.364137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:59.369493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:59.369493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:59.386903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:04:59.389427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:04:59.390283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:59.451367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1041/2069] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:04:59.455726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93834c4ca6e345fb8e2218a207dcc17e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:04:59.462273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:59.463475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1029/2070] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:04:58 2022] DELETE /identity/v3/users/93834c4ca6e345fb8e2218a207dcc17e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:04:59.464209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:04:59.524972 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663eb6-08dc-45b5-b654-82ca59e89b91 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:04:59.581784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:04:59.583078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:04:59.583423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:04:59.583800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:04:59.656037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:04:59.657554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:04:59.667376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1042/2071] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:04:59.994733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:04:59.994733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.009150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc0065-df37-43b5-8bc6-5d96cdeafa3f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.014055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.014710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.014999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.015274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] PATH_INFO: `/v3/projects/c744b363c30a4bb4926a1e03b8f4f682` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.016554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c744b363c30a4bb4926a1e03b8f4f682)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.018560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.187641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.187641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.196361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c744b363c30a4bb4926a1e03b8f4f682', 'request_id': 'req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.202756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560b6f-8dd0-45a6-8b29-17455249a7bd service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.202906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c744b363c30a4bb4926a1e03b8f4f682', 'request_id': 'req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.216246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:00.270567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.271753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.272109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.272384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] PATH_INFO: `/v3/projects/2b7f7b3653564f87b650fc826da1dfde` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.274111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7f7b3653564f87b650fc826da1dfde)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.276978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.341674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c744b363c30a4bb4926a1e03b8f4f682', 'request_id': 'req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.352861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] The token cache is being invalidate because project c744b363c30a4bb4926a1e03b8f4f682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:00.353223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.353820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17bc8d1f-8d9b-4e0f-b41b-75af2e2b3346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.355328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1030/2072] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:04:59 2022] DELETE /identity/v3/projects/c744b363c30a4bb4926a1e03b8f4f682 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:00.361205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12849532-bcd1-4868-93a9-0f6b0732f8b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.361547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12849532-bcd1-4868-93a9-0f6b0732f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.361823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12849532-bcd1-4868-93a9-0f6b0732f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.383203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-12849532-bcd1-4868-93a9-0f6b0732f8b2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:00.384453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-12849532-bcd1-4868-93a9-0f6b0732f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.397470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7f7b3653564f87b650fc826da1dfde', 'request_id': 'req-a754a3c1-0da1-4965-881d-a24fb5eeae79'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.425796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7f7b3653564f87b650fc826da1dfde', 'request_id': 'req-a754a3c1-0da1-4965-881d-a24fb5eeae79'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.477869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1031/2073] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:00.486183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.486537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.486885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.508677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:00.509984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.517729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1032/2074] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:00.521921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.530693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0130a3-c088-45d3-9413-6a2ee4ed4efd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.535475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.536066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.536340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.536624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.537491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:00.537796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.540049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.551828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1033/2075] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:05:00.556921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.567173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfe65e-49f0-4f38-869c-7cab06c2d739 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.572001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.572999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.573506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.573925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.574901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.577799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.581687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7f7b3653564f87b650fc826da1dfde', 'request_id': 'req-a754a3c1-0da1-4965-881d-a24fb5eeae79'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.604716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] The token cache is being invalidate because project 2b7f7b3653564f87b650fc826da1dfde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:00.605229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.606515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a754a3c1-0da1-4965-881d-a24fb5eeae79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:00.608389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1043/2076] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:00 2022] DELETE /identity/v3/projects/2b7f7b3653564f87b650fc826da1dfde => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:00.619386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530acf8-ce19-4af6-bcaf-24f4257db228 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.619671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530acf8-ce19-4af6-bcaf-24f4257db228 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.619996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530acf8-ce19-4af6-bcaf-24f4257db228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.641323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7530acf8-ce19-4af6-bcaf-24f4257db228 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:00.642521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7530acf8-ce19-4af6-bcaf-24f4257db228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.652727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1044/2077] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:00.660285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.660678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.660842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.682427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:00.683706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.692260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1045/2078] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:00.697251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.712095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536e53c-d635-4a3b-bfa7-fd39c54512c2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.714408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.717241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.717241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.717241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.720103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:00.720735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.728123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.749270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1034/2079] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:00.755350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.758137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:00.759925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1046/2080] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:05:00.763179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.788183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2ccc3-cc23-42b3-be76-ab268a3a06c9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.857484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaaf33-c263-431c-a88d-a6843bb01344 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:00.863936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.863936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.863936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.863936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.868631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.868631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:00.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:00.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:00.869160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:00.869759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:00.870788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:00.971937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1035/2081] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:00.978207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.978578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:00.988175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1047/2082] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:00.989793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:00.990995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f7776-e66d-4ea9-b7d3-4f617f1f8553 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.050833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.051350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.051627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.051934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.052508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dcd69-acea-4b24-b8f3-8b2eb8950b99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.052647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.054167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.057892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:01.058032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.058504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.058780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.059078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.059724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.061811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.066171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1036/2083] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:01.070473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.079900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de355-dad8-4b81-9674-ed89ffe2a951 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.084445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.085029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.085289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.086104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.086702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.088297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.098438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1037/2084] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:01.102503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.111623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f902a-326a-4d7b-9551-53a836981455 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.116455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.117619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.118087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.118496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] PATH_INFO: `/v3/projects/8f0a40640e574294870ad343abe6dffc/users/eb8b473bbb854f449e248bed27df6fba/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.136890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0a40640e574294870ad343abe6dffc, user_id=eb8b473bbb854f449e248bed27df6fba, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.139785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a149bf-1b31-4b07-8285-42d795f4e299 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.251426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1048/2085] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:00 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:01.255212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.265076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477b162-d7ab-42f3-9394-b6680e0f0d37 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.271046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.272028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.272291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.272575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.273216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.274837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.287833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1049/2086] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:01.292173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.298995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1038/2087] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:01 2022] PUT /identity/v3/projects/8f0a40640e574294870ad343abe6dffc/users/eb8b473bbb854f449e248bed27df6fba/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:01.300434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.300719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.300996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.305155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe172e0f-1612-4ec0-8690-7cacfb334751 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.323990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:01.325353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:01.366416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.367641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.367950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.368227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.368869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.370489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.381065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1050/2088] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:01.385370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.387577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1039/2089] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:01.398007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.399521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:01.401375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68ca3-a310-4a4a-8ede-072754136afe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.406613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.407280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.407563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.407872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] PATH_INFO: `/v3/projects/23c8a1db00944c59950120c030ab47e3/users/f2e7493d38d24493b3298ac37592ecd4/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.423848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c8a1db00944c59950120c030ab47e3, user_id=f2e7493d38d24493b3298ac37592ecd4, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.426529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.457187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4c5ad-f7bd-424e-8893-a0e20c1d63af None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.461187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.461805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.462086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.462363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.471460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.473561 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada15c-fe14-4b47-8c40-54d3b44f1011 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.484200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1040/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:01.553127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1051/2091] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:01 2022] PUT /identity/v3/projects/23c8a1db00944c59950120c030ab47e3/users/f2e7493d38d24493b3298ac37592ecd4/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:01.558581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b552a5-d955-4c0d-b56e-34019fa9d643 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.558874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b552a5-d955-4c0d-b56e-34019fa9d643 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.559155 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b552a5-d955-4c0d-b56e-34019fa9d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.591510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41b552a5-d955-4c0d-b56e-34019fa9d643 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:01.592797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41b552a5-d955-4c0d-b56e-34019fa9d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:01.659590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1041/2092] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:01.671431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:01.673408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:01.732712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b969e-ff33-458c-aa96-53a0175e8ab1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:01.736391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:01.736989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:01.737263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:01.737535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:01.746674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:01.748238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:01.758696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1052/2093] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:01.836843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:01.925097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:02.535271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:02.855965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:03.386669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd613a0f-85f1-487a-8ace-c9d94ce3982e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.387081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd613a0f-85f1-487a-8ace-c9d94ce3982e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.387463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd613a0f-85f1-487a-8ace-c9d94ce3982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.427256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fd613a0f-85f1-487a-8ace-c9d94ce3982e None None] MFA Rules not processed for user `eb8b473bbb854f449e248bed27df6fba`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:03.428330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd613a0f-85f1-487a-8ace-c9d94ce3982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:03.502493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1042/2094] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:03.507567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:03.517340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936d933-7079-4882-86a2-87754e883f49 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:03.532420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:03.533259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.533895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.534381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.535169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:03.537307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:03.548105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:03.647278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.647278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.647278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.647692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1053/2095] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:03 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:03.648155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:03.689527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] MFA Rules not processed for user `f2e7493d38d24493b3298ac37592ecd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:03.690940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:03.732221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba390d-8ce1-4a2f-aca2-a5184850a718 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:03.737608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:03.738225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.738499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.738767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.739396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:03.741395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:03.770942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1043/2096] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:03.784483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:03.785947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:03.851231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1054/2097] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:03 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:03.855245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:03.863813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c88490-9de9-4780-9818-bd700eb995a9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:03.866196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679354f7-0037-45a9-9576-39b409c1b126 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:03.869028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:03.869905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.870394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.870900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.872830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:03.873443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.873726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.874004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.874657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:03.876327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:03.882696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:03.887281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:03.889000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:03.901650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1055/2098] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:03.903841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:03.920552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1044/2099] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:05:03.927671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a629e-bc3a-47e8-987d-0866c8a4c0a3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:03.937603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:03.938305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:03.938586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:03.938862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] PATH_INFO: `/v3/projects/daa967204f054d10bb484d8db013b8ea/users/fe3f8bb053904619b9966005e0c0543c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:03.971681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa967204f054d10bb484d8db013b8ea, user_id=fe3f8bb053904619b9966005e0c0543c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:03.974976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:04.102934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1056/2100] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:03 2022] PUT /identity/v3/projects/daa967204f054d10bb484d8db013b8ea/users/fe3f8bb053904619b9966005e0c0543c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:04.106789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:04.118742 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93ab6-0344-44b0-9463-ae898ce99a89 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:04.192531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:04.193183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:04.193467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:04.193747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:04.194454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:04.196105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b14ae-e6f1-4bbf-97d8-1fb250a5b26e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:04.206877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1045/2101] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:04.212698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:04.223100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa00f-8958-4e86-a887-452774747d3f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:04.228358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:04.229006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:04.229285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:04.229554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] PATH_INFO: `/v3/projects/daa967204f054d10bb484d8db013b8ea/users/fe3f8bb053904619b9966005e0c0543c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:04.231500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa967204f054d10bb484d8db013b8ea, user_id=fe3f8bb053904619b9966005e0c0543c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:04.234173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10add331-0cca-4bad-98cc-82d4318f013f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:04.245071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1057/2102] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:04 2022] PUT /identity/v3/projects/daa967204f054d10bb484d8db013b8ea/users/fe3f8bb053904619b9966005e0c0543c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:05.744448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:05.872716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:05.873000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:05.876971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:05.878087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122501-b790-4f08-90bc-af8cbed14c1d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:05.878516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122501-b790-4f08-90bc-af8cbed14c1d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:05.878642 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122501-b790-4f08-90bc-af8cbed14c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:05.909287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] MFA Rules not processed for user `f2e7493d38d24493b3298ac37592ecd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:05.910629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:05.924042 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34122501-b790-4f08-90bc-af8cbed14c1d None None] MFA Rules not processed for user `fe3f8bb053904619b9966005e0c0543c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:05.925521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34122501-b790-4f08-90bc-af8cbed14c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:05.985814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1046/2103] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:06.015621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1058/2104] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:06.031785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:06.061923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:06.139908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55819b15-4000-49a0-9186-a5db31e1857b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:06.143929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:06.144646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:06.145026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:06.145412 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:06.254103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:06.255576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903fe9e-13db-48c3-bb4c-1f4e6f8336dd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:06.271590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1047/2105] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:07.159062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:07.159543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:07.159655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:07.161903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1059/2106] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:05:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:05:07.210428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:07.210939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:07.211019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:07.231938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:07.233197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:07.298567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1048/2107] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:07.310779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:07.313058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:07.331924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:07.333365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:07.336180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f23d15-65e8-43c3-b55e-fc3039152fec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:07.342184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:07.343025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:07.344754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:07.344754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:07.352838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:07.354435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:07.364410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1060/2108] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:07.405753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ad22e-3355-4428-a05b-1013c387ec1d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:07.412662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:07.413319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:07.413602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:07.413887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] PATH_INFO: `/v3/users/2d6fc583bb8a4434b0c6538c9ffbde75` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:07.415008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6fc583bb8a4434b0c6538c9ffbde75)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:07.416945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:07.433353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:07.698486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6fc583bb8a4434b0c6538c9ffbde75'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:07.699251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6fc583bb8a4434b0c6538c9ffbde75'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:07.706666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6fc583bb8a4434b0c6538c9ffbde75'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:07.718313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6fc583bb8a4434b0c6538c9ffbde75'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:07.764376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:07.798418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6fc583bb8a4434b0c6538c9ffbde75'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:07.805613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1049/2109] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:07 2022] DELETE /identity/v3/users/2d6fc583bb8a4434b0c6538c9ffbde75 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:08.120295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.133031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999650e-d53c-4856-a66f-f4592e4e06cd service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:08.197300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.198763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:08.201879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:08.202510 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.202788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.203061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] PATH_INFO: `/v3/projects/cbb076c769694966bbfea2f23ea8b8a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.204220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb076c769694966bbfea2f23ea8b8a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:08.209704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:08.259675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee02-abc6-4860-b731-fa2c7cfc52b7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:08.263234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:08.263859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.264135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.264400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.266720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:08.330972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:08.332528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54cb80-dbd7-4f34-9607-f9c21cc2a808 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:08.342417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1050/2110] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:08.353588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb076c769694966bbfea2f23ea8b8a4', 'request_id': 'req-b124aacc-dc62-4c7a-a6cb-72035aabb77a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:08.360432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb076c769694966bbfea2f23ea8b8a4', 'request_id': 'req-b124aacc-dc62-4c7a-a6cb-72035aabb77a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:08.466807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb076c769694966bbfea2f23ea8b8a4', 'request_id': 'req-b124aacc-dc62-4c7a-a6cb-72035aabb77a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:08.478275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] The token cache is being invalidate because project cbb076c769694966bbfea2f23ea8b8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:08.478600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:08.479226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b124aacc-dc62-4c7a-a6cb-72035aabb77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:08.480831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1061/2111] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:08 2022] DELETE /identity/v3/projects/cbb076c769694966bbfea2f23ea8b8a4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:08.486190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.486483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.486758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.507324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:08.508570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:08.572379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1051/2112] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:08.580317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.580601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.580877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.611862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:08.613085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:08.621674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1062/2113] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:08.626041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.636618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ee70-7a06-47d3-bf68-14846bd854e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:08.641257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:08.644514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.644514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.644514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.644514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:08.644514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:08.645584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:08.657580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1052/2114] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:05:08.662052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.673314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb913721-a2b0-46f6-a2e3-88586cf9a4a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:08.678045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:08.682833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.682833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.682833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.682833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:08.682833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:08.816717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1063/2115] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:08 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:08.821480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.832941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758955e-1903-4e22-9727-6c0be565d18d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:08.891931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:08.894882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:08.894882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:08.894882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:08.894882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:08.899306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:08.985299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1053/2116] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:08 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:08.986956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:08.997569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47481bb-c5e3-41a8-be57-8f717f74c010 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.002347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.002943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.003273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.003484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.004169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.005758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.016631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1064/2117] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:09.020730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.031557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35603a-12f7-4c2f-be8a-2759e5a0ceb1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.036298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.036889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.037165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.037439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.038217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.039824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bd540-0493-42e8-b947-07a09d28a45b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.050738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1054/2118] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:09.057069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.068172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bc86d-fe02-421d-a7b7-6fd135308e59 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.073570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.074300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.074771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.075048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] PATH_INFO: `/v3/projects/b533aabc185e464eb2bb1abb5927d180/users/6d2e26de98574a349c2acdcb713e1891/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.093816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b533aabc185e464eb2bb1abb5927d180, user_id=6d2e26de98574a349c2acdcb713e1891, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.096557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.209994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1065/2119] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:09 2022] PUT /identity/v3/projects/b533aabc185e464eb2bb1abb5927d180/users/6d2e26de98574a349c2acdcb713e1891/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:09.225942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.226314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.227318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.258894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] MFA Rules not processed for user `6d2e26de98574a349c2acdcb713e1891`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:09.260194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:09.341594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1055/2120] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:09.348716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.416242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982597ba-0390-4ef8-9087-ff209a077ecc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.421017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.421625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.421904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.422177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.422871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.425089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.467270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:09.506361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1066/2121] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:09 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:09.510914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.522377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab83ce-77da-4008-b947-9a9cadbb9605 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.584638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.585113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.585395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.585670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.586309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.588568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.688017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1056/2122] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:09 2022] POST /identity/v3/users => generated 295 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:09.689199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.700186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84332-0362-46c1-86c6-b54db482b8d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.704807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.705399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.705681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.705953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.706595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.708240 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.718536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1067/2123] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:09.722825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.733554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d1678-0d82-4558-8c03-07b1899198a0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.738208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.738846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.739126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.739396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] PATH_INFO: `/v3/projects/b960acf7c9b848ab8c886e49165e4367/users/b98b32de35164b52beaa06bd8b664a59/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.756247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b960acf7c9b848ab8c886e49165e4367, user_id=b98b32de35164b52beaa06bd8b664a59, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.759048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.769567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:09.821877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1057/2124] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:09 2022] PUT /identity/v3/projects/b960acf7c9b848ab8c886e49165e4367/users/b98b32de35164b52beaa06bd8b664a59/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:09.825967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.836294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302139-14c1-460a-bbb1-e1942629dba7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.893009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.893673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.893961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.894280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.894945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.896624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02e12f-75c1-41b7-bc62-0fc9e8def79a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.907638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1068/2125] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:09.912056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:09.922692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348b875-c569-4af6-adc1-5b27c7554c6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:09.927347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:09.928842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.928971 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.929260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] PATH_INFO: `/v3/projects/b960acf7c9b848ab8c886e49165e4367/users/b98b32de35164b52beaa06bd8b664a59/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.931282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b960acf7c9b848ab8c886e49165e4367, user_id=b98b32de35164b52beaa06bd8b664a59, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:09.934947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:09.946268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1058/2126] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:09 2022] PUT /identity/v3/projects/b960acf7c9b848ab8c886e49165e4367/users/b98b32de35164b52beaa06bd8b664a59/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:09.957961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:09.958337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:09.958621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:09.993691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] MFA Rules not processed for user `b98b32de35164b52beaa06bd8b664a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:09.994892 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:10.080847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1069/2127] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:10.101799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:10.103722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:10.169839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1663aaa-91be-4491-ae4a-7a0a8c773155 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:10.177432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:10.178045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:10.178929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:10.179623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:10.247119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:10.248609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1249b-f57a-4278-ae26-7cf4ade2fd34 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:10.263407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1059/2128] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:11.295701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:11.609803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:13.043640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:13.397331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:13.706279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.232150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.637675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.666295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.723217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.751941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.780499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.866671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:14.895397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:15.220385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:16.934830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:16.936300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:16.949558 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cdb25-1ce5-4541-bbb1-9fad0e56f0f3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:16.953273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:16.953870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:16.954144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:16.954412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:16.963185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:16.964893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e65bbf-88ac-40fa-91eb-25ff8907bad4 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:16.975167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1070/2129] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:05:25.306464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:26.970343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:31.145101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:31.148089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:31.148089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:31.148089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1060/2130] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:05:31.172579 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:31.173201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:31.173201 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:31.193897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] MFA Rules not processed for user `25af3fcbdf174cc2bf22fb99c83575f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:31.195292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:31.257563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1071/2131] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:31.271050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:31.273052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:31.291265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f5036-5d39-436f-9286-a3096da0b7a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:31.298021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:31.298632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:31.298907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:31.299251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:31.309223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:31.310818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:31.325591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1061/2132] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:32.110978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:32.112626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:32.221188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee95ed-cd50-46a4-af9b-890abc31b219 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:32.226565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:32.227437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:32.227720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:32.228018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] PATH_INFO: `/v3/users/eb8b473bbb854f449e248bed27df6fba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:32.229309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8b473bbb854f449e248bed27df6fba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:32.231251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:32.561919 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8b473bbb854f449e248bed27df6fba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:32.562662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8b473bbb854f449e248bed27df6fba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:32.575638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8b473bbb854f449e248bed27df6fba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:32.687424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8b473bbb854f449e248bed27df6fba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:32.695536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8b473bbb854f449e248bed27df6fba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:32.701496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1072/2133] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:32 2022] DELETE /identity/v3/users/eb8b473bbb854f449e248bed27df6fba => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:33.027081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:33.050213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16811-d867-4635-9e46-74966d1a99b3 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:33.115088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:33.115717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:33.116078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:33.116354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] PATH_INFO: `/v3/projects/8f0a40640e574294870ad343abe6dffc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:33.117468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0a40640e574294870ad343abe6dffc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:33.119355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:33.288209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0a40640e574294870ad343abe6dffc', 'request_id': 'req-f2d70981-705e-424c-b162-33b726601522'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.294476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0a40640e574294870ad343abe6dffc', 'request_id': 'req-f2d70981-705e-424c-b162-33b726601522'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.413369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0a40640e574294870ad343abe6dffc', 'request_id': 'req-f2d70981-705e-424c-b162-33b726601522'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.428144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] The token cache is being invalidate because project 8f0a40640e574294870ad343abe6dffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:33.428509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.429168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.431224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1062/2134] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:33 2022] DELETE /identity/v3/projects/8f0a40640e574294870ad343abe6dffc => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:33.435181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:33.436759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:33.452545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8061d19-32c3-46fb-b71c-74b2c66a42ac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:33.464916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:33.527580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:33.528283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:33.528568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:33.528906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] PATH_INFO: `/v3/users/fe3f8bb053904619b9966005e0c0543c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:33.529973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3f8bb053904619b9966005e0c0543c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:33.532620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:33.753223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3f8bb053904619b9966005e0c0543c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.753881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3f8bb053904619b9966005e0c0543c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.766133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3f8bb053904619b9966005e0c0543c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.847106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3f8bb053904619b9966005e0c0543c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.854972 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2c814f75-88f3-4a4b-bb51-b7ed5a545bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3f8bb053904619b9966005e0c0543c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:33.861120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1073/2135] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:33 2022] DELETE /identity/v3/users/fe3f8bb053904619b9966005e0c0543c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:34.111344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.112930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:34.125811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d70981-705e-424c-b162-33b726601522 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.184456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.185077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.185354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.185623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] PATH_INFO: `/v3/projects/daa967204f054d10bb484d8db013b8ea` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.186711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa967204f054d10bb484d8db013b8ea)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.188791 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:34.211873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:34.316031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa967204f054d10bb484d8db013b8ea', 'request_id': 'req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:34.322657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa967204f054d10bb484d8db013b8ea', 'request_id': 'req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:34.424221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa967204f054d10bb484d8db013b8ea', 'request_id': 'req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:34.435006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] The token cache is being invalidate because project daa967204f054d10bb484d8db013b8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:34.439881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:34.439881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7c2ec9d-860d-4c79-bf3d-3f1ee7df4c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:34.439881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1063/2136] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:34 2022] DELETE /identity/v3/projects/daa967204f054d10bb484d8db013b8ea => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:34.443545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.444067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.444168 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.464321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:34.465526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:34.524356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1074/2137] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:34.531973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.532219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.532495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.552634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:34.553823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:34.561890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1064/2138] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:34.566176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.576477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410003a-08bd-4f10-b5e8-b5118247e8b5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.581064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.581667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.581948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.582221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.582872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:34.583175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.585238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:34.601295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1075/2139] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:05:34.606660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.616300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b964f97-cc7f-45cb-ba3b-2a9ba41884c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.620909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.621521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.621799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.622066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.622681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.624593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:34.762197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1065/2140] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:34 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:34.766383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.777020 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cecc69-bcc0-440b-8573-ca97ec26bb32 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.795528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:34.831629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.832262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.833033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.833303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.833925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.835890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:34.933369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1076/2141] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:34 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:34.937651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.948428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef7029-3786-4436-8ccf-a120be0e90b1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.953199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.953786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.954147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.954430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.955068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.956698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:34.968090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1066/2142] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:34.971966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:34.981853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b48aa-a544-4362-b515-157ed5976b99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:34.986603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:34.987208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:34.987486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:34.987770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:34.988426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:34.990040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d3527-d1e3-4452-8318-226c3b7850e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:35.003958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1077/2143] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:35.005491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:35.015607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983371b-0257-4c53-a1b7-d8c6d0afd0e6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:35.020729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:35.021360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:35.021641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:35.021912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] PATH_INFO: `/v3/projects/df09006c5e41417b938989b606460844/users/a772084ab01e4323bd4267e4b4996ad8/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:35.036064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df09006c5e41417b938989b606460844, user_id=a772084ab01e4323bd4267e4b4996ad8, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:35.038620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:35.150861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1067/2144] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:35 2022] PUT /identity/v3/projects/df09006c5e41417b938989b606460844/users/a772084ab01e4323bd4267e4b4996ad8/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:35.161016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:35.161303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:35.161605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:35.188750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] MFA Rules not processed for user `a772084ab01e4323bd4267e4b4996ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:35.189957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:35.256058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1078/2145] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:35.275526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:35.277287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:35.338319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901f35-006a-45ec-8afb-4679176acf61 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:35.342287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:35.342916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:35.343197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:35.343471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:35.351608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:35.353054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d9eaa-2833-4ca7-bc42-7404709a2080 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:35.362126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1068/2146] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:37.376898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:37.380955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:37.442860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd1ed5-438c-47aa-bb9e-32028353ad59 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:37.447387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:37.448052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:37.448331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:37.448600 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] PATH_INFO: `/v3/users/93ab54fa81504882967051ea59092aa4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:37.449693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ab54fa81504882967051ea59092aa4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:37.451635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:37.627063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:37.649898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ab54fa81504882967051ea59092aa4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:37.650608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ab54fa81504882967051ea59092aa4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:37.669923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ab54fa81504882967051ea59092aa4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:37.750455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ab54fa81504882967051ea59092aa4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:37.758601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ab54fa81504882967051ea59092aa4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:37.767537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1079/2147] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:37 2022] DELETE /identity/v3/users/93ab54fa81504882967051ea59092aa4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:37.773377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:37.773861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:37.773954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:37.796998 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:37.798238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:37.849142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:37.867488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1069/2148] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:37.877441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:37.878849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:37.947134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc32bb-a16c-4c25-af79-c3265fac2605 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:37.950737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:37.951341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:37.951619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:37.951988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:37.965280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:37.967086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61661cb9-1672-4a7d-9de0-3379568d15ca service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:37.985455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1080/2149] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:38.480282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:38.482272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:38.498041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73282c03-e3ce-4ca6-8bae-b01b9942a8e6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:38.503022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:38.503731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:38.504717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:38.505078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] PATH_INFO: `/v3/projects/232f67faed57453e99c930bd1fa67279` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:38.506281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=232f67faed57453e99c930bd1fa67279)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:38.508600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:38.653528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '232f67faed57453e99c930bd1fa67279', 'request_id': 'req-83e0e013-78a0-415f-bf34-5072860c2e05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:38.659882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '232f67faed57453e99c930bd1fa67279', 'request_id': 'req-83e0e013-78a0-415f-bf34-5072860c2e05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:38.744493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '232f67faed57453e99c930bd1fa67279', 'request_id': 'req-83e0e013-78a0-415f-bf34-5072860c2e05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:38.755839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] The token cache is being invalidate because project 232f67faed57453e99c930bd1fa67279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:38.756065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:38.756741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83e0e013-78a0-415f-bf34-5072860c2e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:38.758238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1070/2150] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:38 2022] DELETE /identity/v3/projects/232f67faed57453e99c930bd1fa67279 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:38.763688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:38.764274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:38.764274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:38.784136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:38.785983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:38.866111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1081/2151] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:38.868296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:38.869155 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:38.869563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:38.911075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:38.911075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:38.911075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1071/2152] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:38.915405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:38.925562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899675f-1139-471c-8d73-c615e6b67f31 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:38.946218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:38.946871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:38.947151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:38.947453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:38.948131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:38.948442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:38.950154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:38.963169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1082/2153] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:05:38.967584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:38.977813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddde5e-d1e8-49b1-bf23-43bebe38987c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:38.982805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:38.983419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:38.983692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:38.984019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:38.984702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:38.986769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.156765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1072/2154] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:38 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:39.162047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:39.176067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad564396-a927-4152-9513-b9d1c96c6bac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:39.216830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:39.235206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:39.235841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.236122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.236394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.237008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:39.238942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.353716 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1083/2155] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:39 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:39.358530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:39.368847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264312c-f17a-46ea-bc45-253d14d7d6c8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:39.373529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:39.374130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.374413 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.374689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.375329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:39.377080 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.389962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1073/2156] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:39.395774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:39.408499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7daab-dac2-48fd-b59d-8571a528b7b3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:39.426415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:39.426415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.426694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.428430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.428430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:39.431163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16186c-4535-43db-a4a5-a6f3d56ef74f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.446443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1084/2157] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:39.447591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:39.461001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b586335-fa26-44c5-abe5-1a5a5c63e6d9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:39.466382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:39.467025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.467317 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.467596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] PATH_INFO: `/v3/projects/233f1c55781b4099be6e250e664c4fb9/users/dfaf5a716c5b448f8e1274de2fad307c/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.484483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] RBAC: Authorizing `identity:create_grant(project_id=233f1c55781b4099be6e250e664c4fb9, user_id=dfaf5a716c5b448f8e1274de2fad307c, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:39.487198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.578899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1074/2158] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:39 2022] PUT /identity/v3/projects/233f1c55781b4099be6e250e664c4fb9/users/dfaf5a716c5b448f8e1274de2fad307c/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:39.584140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d04c-ca7a-45af-81b3-571757160bb0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.584427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d04c-ca7a-45af-81b3-571757160bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.584727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d04c-ca7a-45af-81b3-571757160bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.603946 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b4a4d04c-ca7a-45af-81b3-571757160bb0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:39.605202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b4a4d04c-ca7a-45af-81b3-571757160bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:39.666122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1085/2159] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:39.676751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:39.679575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:39.738365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc589-31ba-4e6e-8662-be8235b2f3de admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:39.742558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:39.743173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:39.743444 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:39.743710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:39.753120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:39.754531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:39.763903 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1075/2160] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:45.887571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:45.888144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:45.888224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:45.918423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] MFA Rules not processed for user `dfaf5a716c5b448f8e1274de2fad307c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:45.920720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:45.998668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1086/2161] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:46.002922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.014184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98b266-99ae-4c9e-aa7c-ac520122da7d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.019865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.020490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.020762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.021219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.021857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.023840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.030592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:46.132353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1076/2162] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:46 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:46.133747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.146291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccef3f-a4bb-45ba-8083-7d71a66973e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.210468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:46.222952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.223595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.223914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.224215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.224839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.226804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.310184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1087/2163] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:46 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:46.319635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.329850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aac11-314c-4863-92d4-21d5cc28cc6e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.334390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.334979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.335258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.335557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.336289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.337960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.348437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1077/2164] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:46.354327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.364750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1cefe-b773-495d-b19c-17c642e4dd37 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.369505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.370499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.370781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.371056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] PATH_INFO: `/v3/projects/530302527b314777b0ac3241328098e6/users/4b403696590747a9959a9321b259b1f3/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.386802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] RBAC: Authorizing `identity:create_grant(project_id=530302527b314777b0ac3241328098e6, user_id=4b403696590747a9959a9321b259b1f3, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.389574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.527887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1088/2165] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:46 2022] PUT /identity/v3/projects/530302527b314777b0ac3241328098e6/users/4b403696590747a9959a9321b259b1f3/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:46.531859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.542324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff9a2f-8ea7-4fc3-a6e3-e4a0ad903ab2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.601693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.602308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.602595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.602853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.604019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.608869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.617932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1078/2166] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:46.622092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:46.639172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc33bfb-93fc-426d-b733-4b3fbfb605be admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:46.639172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:46.639795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:46.640063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:46.640342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] PATH_INFO: `/v3/projects/530302527b314777b0ac3241328098e6/users/4b403696590747a9959a9321b259b1f3/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:46.642311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=530302527b314777b0ac3241328098e6, user_id=4b403696590747a9959a9321b259b1f3, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:46.645202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9da81-9c80-4ba6-94dc-1f964a853adc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:46.655839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1089/2167] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:46 2022] PUT /identity/v3/projects/530302527b314777b0ac3241328098e6/users/4b403696590747a9959a9321b259b1f3/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:48.634780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:48.648072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:48.706737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6485cd9-2ced-4782-8ce6-1a3f743ca937 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:48.842702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:48.843369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:48.843664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:48.843995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] PATH_INFO: `/v3/users/a772084ab01e4323bd4267e4b4996ad8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:48.845267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a772084ab01e4323bd4267e4b4996ad8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:48.848318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:49.197014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a772084ab01e4323bd4267e4b4996ad8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:49.204128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a772084ab01e4323bd4267e4b4996ad8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:49.220602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a772084ab01e4323bd4267e4b4996ad8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:49.258666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a772084ab01e4323bd4267e4b4996ad8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:49.260859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:49.412005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a772084ab01e4323bd4267e4b4996ad8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:49.428059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1079/2168] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:48 2022] DELETE /identity/v3/users/a772084ab01e4323bd4267e4b4996ad8 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:49.432742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff06ce9-8059-4f47-ae1b-e14c9855ffed None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:49.433835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff06ce9-8059-4f47-ae1b-e14c9855ffed None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:49.434310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff06ce9-8059-4f47-ae1b-e14c9855ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:49.480410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fff06ce9-8059-4f47-ae1b-e14c9855ffed None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:49.482018 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fff06ce9-8059-4f47-ae1b-e14c9855ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:49.586176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1090/2169] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:49.613408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:49.615726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:49.768287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4258b7-28c7-4b3d-b67f-c2cf0f457992 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:49.773730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:49.774923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:49.775214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:49.775487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:49.813688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:49.816781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:49.853787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1080/2170] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:05:50.739448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59ba65-44f2-4c80-ba5b-ba7912b19c59 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:50.739979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59ba65-44f2-4c80-ba5b-ba7912b19c59 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:50.740186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59ba65-44f2-4c80-ba5b-ba7912b19c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:50.765226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ec59ba65-44f2-4c80-ba5b-ba7912b19c59 None None] MFA Rules not processed for user `684d59041e82466d8aad6d2964842bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:50.766581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ec59ba65-44f2-4c80-ba5b-ba7912b19c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:50.840158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1091/2171] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:50.860626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:50.865262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:50.893305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b158e-0c4d-43ec-8e64-02f02a7cbb21 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:50.907416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:50.908253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:50.909396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:50.910808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] PATH_INFO: `/v3/projects/df09006c5e41417b938989b606460844` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:50.912422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df09006c5e41417b938989b606460844)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:50.914836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:51.128459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df09006c5e41417b938989b606460844', 'request_id': 'req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:51.139239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df09006c5e41417b938989b606460844', 'request_id': 'req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:51.236187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df09006c5e41417b938989b606460844', 'request_id': 'req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:51.253203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] The token cache is being invalidate because project df09006c5e41417b938989b606460844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:51.253499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:51.254300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ab03a72a-1bb0-45a9-8fc1-75714ed8e6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:51.256324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1081/2172] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:50 2022] DELETE /identity/v3/projects/df09006c5e41417b938989b606460844 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:51.261460 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:51.261825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:51.262195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:51.310769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:51.312864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:51.382594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:51.410593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1092/2173] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:51.417915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:51.418268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:51.418610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:51.450411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:51.452211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:51.467916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1082/2174] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:51.472387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:51.485156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30d05e-d1ce-44e5-8ac2-ed76ff548f85 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:51.500725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:51.502745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:51.503134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:51.503521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:51.504408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:51.504850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:51.507555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:51.528161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1093/2175] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:05:51.529033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:51.540162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755c1cf-aa78-4e19-a787-5241ed43d040 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:51.548084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:51.550341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:51.550341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:51.550341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:51.551735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:51.554957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:51.718759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1083/2176] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:51 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:51.723270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:51.734740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35366d-bfbb-495f-b087-921e903223a3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:51.813681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:51.815103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:51.815680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:51.816145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:51.816792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:51.818826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:52.201420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.201420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:52.201420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72b72f-e5a0-4953-9e3e-9b85dfcf6b2f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:52.206364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:52.206364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.206364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.206364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] PATH_INFO: `/v3/users/6d2e26de98574a349c2acdcb713e1891` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.208343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2e26de98574a349c2acdcb713e1891)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:52.208343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:52.389140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1094/2177] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:51 2022] POST /identity/v3/users => generated 288 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:52.397389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.402780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9da7-bca6-421f-8f0a-482a1850b6e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:52.407354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:52.407969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.408245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.408511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.409681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:52.411250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:52.421483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1095/2178] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:52.426768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.438008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915a082-50c1-47f8-a711-4e643b1e83f4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:52.442727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:52.443325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.443606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.443941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.444576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:52.446160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:52.456220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1096/2179] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:52.460127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.469523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc463669-409b-4d80-b216-0591ec47cef7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:52.474777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:52.475482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.475780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.482744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] PATH_INFO: `/v3/projects/3f4830ac5aa34ea89344a6939b3e86a3/users/6086ee563ac740968d1de9f9384304dd/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.494819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4830ac5aa34ea89344a6939b3e86a3, user_id=6086ee563ac740968d1de9f9384304dd, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:52.497557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065fae9-77b9-488f-9429-4606a8b14ebc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:52.583565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:DELETE=14,keystone:SELECT=83 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:52.591556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2e26de98574a349c2acdcb713e1891'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:52.592305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2e26de98574a349c2acdcb713e1891'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:52.604697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2e26de98574a349c2acdcb713e1891'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:52.618815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2e26de98574a349c2acdcb713e1891'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:52.657457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1097/2180] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:52 2022] PUT /identity/v3/projects/3f4830ac5aa34ea89344a6939b3e86a3/users/6086ee563ac740968d1de9f9384304dd/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:52.677791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.678370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.678698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.714573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] MFA Rules not processed for user `6086ee563ac740968d1de9f9384304dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:52.715691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:52.731836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9744491-37cb-47c8-934c-bdf9b34020c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2e26de98574a349c2acdcb713e1891'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:52.743319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1084/2181] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:52 2022] DELETE /identity/v3/users/6d2e26de98574a349c2acdcb713e1891 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:52.747457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.748052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.748960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.772486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:52.782132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:52.834086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1098/2182] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:52.838690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.840338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:52.921641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1085/2183] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:52.929046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:52.933563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:52.947637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91099332-5380-4b3a-bc94-2ba25dc0396e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:52.953612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:52.954511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:52.955035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:52.956480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:52.957499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:52.960184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.020284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2cb1-b091-4ca7-ad3d-912716c1e97a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.025265 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.027415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.027752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.029553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.041713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.044308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.058242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1086/2184] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:05:53.130247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1099/2185] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:52 2022] POST /identity/v3/projects => generated 373 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:53.130761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.138700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:53.147474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831c1d9-2af7-4e24-b48f-36ff00da0625 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.291431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.293447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.294108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.295449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.297396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.300923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.398855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1087/2186] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:53 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:53.407238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.433169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142abc5-6213-4724-bc9c-84adfc8138bc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.455245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.455245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.455245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.455245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.455830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.457499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.472754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1100/2187] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:53.478002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.502981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88dbe3-798c-4719-a51d-e73ec28ff099 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.516090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.516775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.517057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.517329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] PATH_INFO: `/v3/projects/daca05c54853412aaa53218cc73c0511/users/3dd058a5d4c64652ba75c44aa3a8ff33/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.549894 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=daca05c54853412aaa53218cc73c0511, user_id=3dd058a5d4c64652ba75c44aa3a8ff33, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.552682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.637799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1088/2188] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:53 2022] PUT /identity/v3/projects/daca05c54853412aaa53218cc73c0511/users/3dd058a5d4c64652ba75c44aa3a8ff33/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:53.642169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.653473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5799fe5-07bc-43c6-9281-f3e0fffe7d14 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.726776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.732752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:53.795499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.796785 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.797245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.800072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.802001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.805609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.822877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1101/2189] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:53.827360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:53.845404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521df42-5a70-460e-87cf-156ae8bc7dc3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.859610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.860550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.860832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.861272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] PATH_INFO: `/v3/projects/daca05c54853412aaa53218cc73c0511/users/3dd058a5d4c64652ba75c44aa3a8ff33/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.867402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daca05c54853412aaa53218cc73c0511, user_id=3dd058a5d4c64652ba75c44aa3a8ff33, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.870231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b0cd-46c1-4651-9267-4950503b2d19 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.873222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdc83b-1458-41d4-897f-41d8c3c3c36c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:53.879697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:53.880492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.880860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.881200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] PATH_INFO: `/v3/projects/b533aabc185e464eb2bb1abb5927d180` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.882400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b533aabc185e464eb2bb1abb5927d180)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:53.884418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1102/2190] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:53 2022] PUT /identity/v3/projects/daca05c54853412aaa53218cc73c0511/users/3dd058a5d4c64652ba75c44aa3a8ff33/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:53.884883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:53.894686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:53.895034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:53.895360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:53.928278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] MFA Rules not processed for user `3dd058a5d4c64652ba75c44aa3a8ff33`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:53.930698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:54.003183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1103/2191] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:54.014358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:54.015720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:54.072015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b533aabc185e464eb2bb1abb5927d180', 'request_id': 'req-dae050a0-5bd1-4eef-9853-8ca1ac48690a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.093169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b533aabc185e464eb2bb1abb5927d180', 'request_id': 'req-dae050a0-5bd1-4eef-9853-8ca1ac48690a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.098408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91359b63-c98e-4e8a-8b2c-af0d87d6b97f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:54.182064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:54.183027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:54.183311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:54.183583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:54.206381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b533aabc185e464eb2bb1abb5927d180', 'request_id': 'req-dae050a0-5bd1-4eef-9853-8ca1ac48690a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.226197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] The token cache is being invalidate because project b533aabc185e464eb2bb1abb5927d180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:54.227929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.230003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.236267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1089/2192] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:53 2022] DELETE /identity/v3/projects/b533aabc185e464eb2bb1abb5927d180 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:54.245095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:54.246097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:54.274516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:54.276392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:54.289735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1104/2193] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:05:54.371876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae050a0-5bd1-4eef-9853-8ca1ac48690a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:54.378682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:54.379392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:54.379678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:54.380107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] PATH_INFO: `/v3/users/b98b32de35164b52beaa06bd8b664a59` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:54.381579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98b32de35164b52beaa06bd8b664a59)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:54.384078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:54.676395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98b32de35164b52beaa06bd8b664a59'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.677189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98b32de35164b52beaa06bd8b664a59'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.695698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98b32de35164b52beaa06bd8b664a59'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.719106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98b32de35164b52beaa06bd8b664a59'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.825010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1e5df9ba-aede-4de4-9aa0-822cf79c9e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98b32de35164b52beaa06bd8b664a59'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:54.834576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1090/2194] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:54 2022] DELETE /identity/v3/users/b98b32de35164b52beaa06bd8b664a59 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:54.900989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:55.354684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:55.713187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:55.730404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0f4f4-4208-4d10-9418-5869e8785d0d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:55.881774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:55.882803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:55.883322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:55.883743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] PATH_INFO: `/v3/projects/b960acf7c9b848ab8c886e49165e4367` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:55.885683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b960acf7c9b848ab8c886e49165e4367)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:55.888132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:56.110837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b960acf7c9b848ab8c886e49165e4367', 'request_id': 'req-0eb27017-fa48-47b5-9545-c8c922578237'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:56.117780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b960acf7c9b848ab8c886e49165e4367', 'request_id': 'req-0eb27017-fa48-47b5-9545-c8c922578237'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:56.171333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b960acf7c9b848ab8c886e49165e4367', 'request_id': 'req-0eb27017-fa48-47b5-9545-c8c922578237'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:56.184431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] The token cache is being invalidate because project b960acf7c9b848ab8c886e49165e4367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:05:56.184724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:56.185388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0eb27017-fa48-47b5-9545-c8c922578237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:56.186944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1105/2195] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:05:55 2022] DELETE /identity/v3/projects/b960acf7c9b848ab8c886e49165e4367 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:56.191900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.192478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.192478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.216521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:56.217740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:56.300882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1091/2196] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:56.309052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.309531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.309909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.336978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:56.338992 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:56.344569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:56.361818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1106/2197] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:56.362495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:56.378871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aaaa9-a0a7-4e45-aa86-93d8d4d6f6f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:56.387779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:56.387779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.387779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.387779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.388154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:05:56.388154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:56.388154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ba5cb-8c19-40f1-808b-6ad5fd9d103b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:56.407795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1092/2198] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:05:56.413726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:56.423598 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e86d-e6a0-422c-b34f-cf6f47c8f979 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:56.429131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:56.430222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.430506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.430832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.431478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:56.433738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:56.482395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.482694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.482955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.514652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] MFA Rules not processed for user `4b403696590747a9959a9321b259b1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:56.516491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:56.612041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1107/2199] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:56 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:56.614418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:56.682313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1093/2200] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:56.726124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:56.727982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:56.732837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450e6f1-e1ef-4f2c-9d32-c6c585aa3594 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:56.766071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:56.767257 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.767871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.769481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.770433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:56.775211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:56.901136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9c257-35fa-488d-ba94-c178e5b95259 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:56.951198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1108/2201] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:56 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:56.955228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:56.957669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:56.958375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.958730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.959078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.961401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:56.967611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73728df3-5564-4949-8551-67372ca6a7f3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:56.973795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:56.974748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:56.975224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:56.975709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:56.976719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:56.978724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:56.992730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1109/2202] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:56.999912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:57.012574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560c6bd-5206-4af3-a7f1-206173e4722a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:57.018312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:57.018925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.019401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.019644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.020921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:57.024185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:57.037872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1110/2203] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:57.039583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:57.049680 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:57.050124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc221c39-f181-4baa-9931-79bbe4c5fb0c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:57.051443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf63a7-1729-49cd-8f96-db38939fee01 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:57.056646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:57.057295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.057571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.057841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] PATH_INFO: `/v3/projects/c0cd1eea732046b2bf7e8764853ad6d8/users/fe9a377646704edb8e389c490bbd0512/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.063849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1094/2204] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:05:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:05:57.077063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0cd1eea732046b2bf7e8764853ad6d8, user_id=fe9a377646704edb8e389c490bbd0512, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:57.079516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:57.213756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1111/2205] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:57 2022] PUT /identity/v3/projects/c0cd1eea732046b2bf7e8764853ad6d8/users/fe9a377646704edb8e389c490bbd0512/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:57.227718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb34c-b28a-40b7-9fdb-a71dd9ecebd8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.228869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb34c-b28a-40b7-9fdb-a71dd9ecebd8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.229215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fb34c-b28a-40b7-9fdb-a71dd9ecebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.305596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-664fb34c-b28a-40b7-9fdb-a71dd9ecebd8 None None] MFA Rules not processed for user `fe9a377646704edb8e389c490bbd0512`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:57.308556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-664fb34c-b28a-40b7-9fdb-a71dd9ecebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:57.336201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:57.444934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:57.459404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:57.467630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1095/2206] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:57.509458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.510808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.511148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.554926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:57.557606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:57.591270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add256-e4dd-4427-aff8-f8805cfe0fdd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:57.613181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:57.613875 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.614160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.616239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.616239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:57.617757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:57.692468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1096/2207] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:05:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:05:57.750400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:57.753018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:57.753954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1112/2208] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:57 2022] POST /identity/v3/projects => generated 385 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:57.758051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:57.833142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893b452-4ccd-47e4-80fa-ecbb55197081 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:57.838004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:57.838775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.839462 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.839842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:57.841103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:57.842726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:57.846066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc30ff-5494-4ab2-ae8d-43f5e6458a5f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:57.977998 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:57.978632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:57.978915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:57.979187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.041653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:58.043316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:58.054108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1097/2209] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:05:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:05:58.059831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1113/2210] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:57 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:05:58.063915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:58.077563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aea18-876c-4f8b-bac4-574a7b0db7bb service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:58.097528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:58.099067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:58.099473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:58.099936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.101070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:58.103053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:58.143431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1098/2211] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:58.147056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:58.161517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755b643-d15a-4b0f-9dd2-988255749431 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:58.175950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:58.176876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:58.177308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:58.177732 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.178540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:58.184827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc615-6380-4f9a-b7b5-ae60cb348c41 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:58.207678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1114/2212] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:58.213510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:58.241525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62e681-bbdf-4932-939b-559be0db7601 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:58.262047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:58.262749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:58.263028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:58.267501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] PATH_INFO: `/v3/projects/143f6bdf3eb8435e978e877fc54a2c90/users/cfbe0287e6b64e59be1263f395e3cf49/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.336096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=143f6bdf3eb8435e978e877fc54a2c90, user_id=cfbe0287e6b64e59be1263f395e3cf49, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:58.339547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:58.433746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1099/2213] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:58 2022] PUT /identity/v3/projects/143f6bdf3eb8435e978e877fc54a2c90/users/cfbe0287e6b64e59be1263f395e3cf49/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:58.434741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:58.435217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:58.435309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.475056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] MFA Rules not processed for user `cfbe0287e6b64e59be1263f395e3cf49`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:05:58.476718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:58.651069 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1115/2214] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:05:58.651617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:58.798096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a019b-8478-42ff-88c7-a75bb43fa2d5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:58.809573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:58.810350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:58.810706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:58.811048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:58.811822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:58.814321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:58.897854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:58.982983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1100/2215] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:05:58 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:58.984440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.000811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.004707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:59.016871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c7ef-f2ca-4fa9-b1bd-79b9eb2ff024 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:59.120520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:05:59.171167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:59.172122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.172402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.172678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.173313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:59.175863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:59.187663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974f969-bbf6-4dac-9cde-3e1560c95fb5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:59.203953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:59.204798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.205548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.205948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] PATH_INFO: `/v3/users/6086ee563ac740968d1de9f9384304dd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.213177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6086ee563ac740968d1de9f9384304dd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:59.214057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:59.292667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1116/2216] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:05:58 2022] POST /identity/v3/users => generated 299 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:05:59.296517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.316574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d044-fbd5-4a14-8a18-1ed803b99943 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:59.333086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:59.334013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.334452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.334882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.335712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:59.338347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:59.365550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1117/2217] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:05:59.369546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.394143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847499b-0236-4241-ba69-d7456085d56f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:59.409916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:59.420602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.420881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.421165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] PATH_INFO: `/v3/projects/89e05f16b68d48158ef0a6b497cc97d8/users/6dbd1e37ab14459d851f4260a5b450f0/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.461471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e05f16b68d48158ef0a6b497cc97d8, user_id=6dbd1e37ab14459d851f4260a5b450f0, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:59.462359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:59.722198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6086ee563ac740968d1de9f9384304dd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:59.724188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6086ee563ac740968d1de9f9384304dd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:59.735289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1118/2218] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:59 2022] PUT /identity/v3/projects/89e05f16b68d48158ef0a6b497cc97d8/users/6dbd1e37ab14459d851f4260a5b450f0/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:05:59.737052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6086ee563ac740968d1de9f9384304dd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:59.738803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.746172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:05:59.761492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6086ee563ac740968d1de9f9384304dd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:59.892078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37cf87-e408-4841-826f-f10dda1a700e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:05:59.899487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:05:59.900294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.900675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.901052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.901810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:05:59.903549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:05:59.920484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47f5aa23-efce-4d02-9b69-846afb2d9cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6086ee563ac740968d1de9f9384304dd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:05:59.924186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1119/2219] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:05:59.929697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1101/2220] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:05:58 2022] DELETE /identity/v3/users/6086ee563ac740968d1de9f9384304dd => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:05:59.929830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:05:59.932392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:05:59.932675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:05:59.932950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:05:59.948695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fcffc-1563-4beb-8764-ab8492388617 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:00.002719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:00.002719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:00.021762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:00.022246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:00.022531 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:00.022813 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] PATH_INFO: `/v3/projects/89e05f16b68d48158ef0a6b497cc97d8/users/6dbd1e37ab14459d851f4260a5b450f0/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:00.025384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e05f16b68d48158ef0a6b497cc97d8, user_id=6dbd1e37ab14459d851f4260a5b450f0, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:00.028676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca0206-76ea-43ed-aab2-dce2164e8a1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:00.045539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1120/2221] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:05:59 2022] PUT /identity/v3/projects/89e05f16b68d48158ef0a6b497cc97d8/users/6dbd1e37ab14459d851f4260a5b450f0/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:00.057417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:00.057754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:00.058054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:00.074249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:00.130512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1102/2222] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:00.142035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:00.147749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:00.147907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] MFA Rules not processed for user `6dbd1e37ab14459d851f4260a5b450f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:00.150621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:00.217796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:00.236423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6456b-0658-4d4d-aa17-ee974f386baa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:00.299083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:00.300106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:00.300106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:00.300278 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:00.303671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1121/2223] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:00.318350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:00.320021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:00.405865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb96e0-06cf-46af-86c2-fa5b6da2c175 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:00.412382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:00.413267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:00.413614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:00.413948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:00.420077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:00.421627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:00.427448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:00.429755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:00.461003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1122/2224] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:06:00.462184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1103/2225] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:00.738275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:01.057797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:01.217880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:01.219361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:01.262123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e1fa5-3565-4b28-a062-cb0c4cb1f01f service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:01.308051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:01.309528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:01.309807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:01.310075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] PATH_INFO: `/v3/projects/3f4830ac5aa34ea89344a6939b3e86a3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:01.313358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4830ac5aa34ea89344a6939b3e86a3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:01.320775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:01.538261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4830ac5aa34ea89344a6939b3e86a3', 'request_id': 'req-83d3c6e7-366d-416c-bf29-5b79fa6671ab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:01.545660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4830ac5aa34ea89344a6939b3e86a3', 'request_id': 'req-83d3c6e7-366d-416c-bf29-5b79fa6671ab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:01.645191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4830ac5aa34ea89344a6939b3e86a3', 'request_id': 'req-83d3c6e7-366d-416c-bf29-5b79fa6671ab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:01.645728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:01.656026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:01.669801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] The token cache is being invalidate because project 3f4830ac5aa34ea89344a6939b3e86a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:01.669801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:01.669801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:01.669801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1123/2226] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:01 2022] DELETE /identity/v3/projects/3f4830ac5aa34ea89344a6939b3e86a3 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:01.671977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:01.683252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e686946-8c89-4fe8-81c8-268dd44a49a6 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:01.786258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:01.789328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:01.789818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:01.790259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] PATH_INFO: `/v3/users/3dd058a5d4c64652ba75c44aa3a8ff33` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:01.792283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd058a5d4c64652ba75c44aa3a8ff33)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:01.807825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:02.028911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd058a5d4c64652ba75c44aa3a8ff33'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:02.029646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd058a5d4c64652ba75c44aa3a8ff33'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:02.040607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd058a5d4c64652ba75c44aa3a8ff33'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:02.079731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:02.082260 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:02.093612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd058a5d4c64652ba75c44aa3a8ff33'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:02.169941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3c6e7-366d-416c-bf29-5b79fa6671ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:02.170898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd058a5d4c64652ba75c44aa3a8ff33'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:02.175083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:02.175704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:02.180139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:02.180139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:02.180139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:02.185933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1104/2227] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:01 2022] DELETE /identity/v3/users/3dd058a5d4c64652ba75c44aa3a8ff33 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:02.251290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:02.256195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5877459-06d3-4ac8-8423-89fefaca2e11 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:02.274450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1124/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:06:02.853902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:02.856448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:02.960657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:03.012648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb869ce-584f-4318-9311-59d652767dcb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:03.018005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:03.018832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:03.019173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:03.019519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] PATH_INFO: `/v3/projects/daca05c54853412aaa53218cc73c0511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:03.021299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daca05c54853412aaa53218cc73c0511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:03.023382 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:03.549489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daca05c54853412aaa53218cc73c0511', 'request_id': 'req-ed351c0c-df92-4860-bdd5-0f9cffb80f10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:03.555944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daca05c54853412aaa53218cc73c0511', 'request_id': 'req-ed351c0c-df92-4860-bdd5-0f9cffb80f10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:03.673341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daca05c54853412aaa53218cc73c0511', 'request_id': 'req-ed351c0c-df92-4860-bdd5-0f9cffb80f10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:03.689667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] The token cache is being invalidate because project daca05c54853412aaa53218cc73c0511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:03.690560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:03.691709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed351c0c-df92-4860-bdd5-0f9cffb80f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:03.693776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1105/2229] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:02 2022] DELETE /identity/v3/projects/daca05c54853412aaa53218cc73c0511 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:03.698868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:03.698868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:03.700980 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:03.747052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:03.748591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:03.861136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1125/2230] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:03.868596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:03.871581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:03.872089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:03.938336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:03.943255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:03.971612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:03.972362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1106/2231] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:03.986461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65ab7e-a688-44dd-bc87-33c83b3fb663 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:03.999405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:03.999969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.000258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.000524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.001183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:06:04.001520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.003239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.025437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1126/2232] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:06:04.029569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:04.053897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e4a1b-f1b7-457a-8180-abcf74e3f304 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:04.070302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:04.070926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.071202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.071676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.072364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.080531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.249171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1107/2233] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:04 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:04.250640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:04.266060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7ca41-a8c9-4212-9aeb-38fa38c65af0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:04.320699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:04.381740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:04.384723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.385397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.386669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.388360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.396534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.586397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1127/2234] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:04 2022] POST /identity/v3/users => generated 287 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:04.592182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:04.601915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed979d9b-3cba-436b-bed4-f3769d082753 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:04.615381 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:04.616310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.622355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.622355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.623947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.625787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.638631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1108/2235] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:04.642343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:04.662137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77ef64-580d-4e6d-a4b1-ef551c3d37dc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:04.667070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:04.667949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.668376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.668795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.669595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.671409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a9ffc-438a-4470-9715-119e1f7c99d3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.686533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1128/2236] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:04.691089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:04.701884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d233a4e-88ed-4196-9d92-c4e5b4e74928 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:04.707907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:04.708602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.708864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.709170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] PATH_INFO: `/v3/projects/83857835f34141f18a89c759631878f6/users/c727b8f265774a4bb03d8425212e7540/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.727968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83857835f34141f18a89c759631878f6, user_id=c727b8f265774a4bb03d8425212e7540, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:04.730909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:04.815206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1109/2237] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:04 2022] PUT /identity/v3/projects/83857835f34141f18a89c759631878f6/users/c727b8f265774a4bb03d8425212e7540/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:04.894015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058700c-0927-467a-9ded-d2f365bac8ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:04.904547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058700c-0927-467a-9ded-d2f365bac8ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:04.904547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058700c-0927-467a-9ded-d2f365bac8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:04.940064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c058700c-0927-467a-9ded-d2f365bac8ff None None] MFA Rules not processed for user `c727b8f265774a4bb03d8425212e7540`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:04.942995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c058700c-0927-467a-9ded-d2f365bac8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:05.061171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1129/2238] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:05.301089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:05.306119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:05.375452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184956d-d1c0-429c-a9c1-c6cf335b5fa5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:05.379691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:05.380336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:05.380621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:05.380897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:05.390610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:05.392682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:05.405059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1110/2239] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:06.211261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:08.567713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c8cf-41c2-4051-9445-7d5feb5232e6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:08.568255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c8cf-41c2-4051-9445-7d5feb5232e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:08.568337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c8cf-41c2-4051-9445-7d5feb5232e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:08.605749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5b05c8cf-41c2-4051-9445-7d5feb5232e6 None None] MFA Rules not processed for user `c727b8f265774a4bb03d8425212e7540`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:08.607026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b05c8cf-41c2-4051-9445-7d5feb5232e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:08.615144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1130/2240] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:09.117302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:09.118751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:09.135784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78df572-960c-4b6d-97bd-eea04090539c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:09.150111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:09.150736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:09.151014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:09.151289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:09.161467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:09.238612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:09.240660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:09.253002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1111/2241] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:11.272320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5211c-19e3-4dbd-b5e5-f795939317d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:11.273313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5211c-19e3-4dbd-b5e5-f795939317d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:11.273847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5211c-19e3-4dbd-b5e5-f795939317d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:11.308121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-10c5211c-19e3-4dbd-b5e5-f795939317d5 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:11.309433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10c5211c-19e3-4dbd-b5e5-f795939317d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:11.404004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1131/2242] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:06:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:11.428176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:11.430483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:11.539682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf63e6-f3f2-415a-9db5-f5e7f0f8b676 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:11.546892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:11.547823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:11.548275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:11.548708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:11.581500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:11.584002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:11.645777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1112/2243] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:06:13.074763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121bee-e331-4934-a220-84908cd4e65a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:13.075314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121bee-e331-4934-a220-84908cd4e65a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:13.075448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121bee-e331-4934-a220-84908cd4e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:13.136837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-32121bee-e331-4934-a220-84908cd4e65a None None] MFA Rules not processed for user `dfaf5a716c5b448f8e1274de2fad307c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:13.138938 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-32121bee-e331-4934-a220-84908cd4e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:13.189747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:13.218796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1132/2244] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:15.507058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:15.509521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:15.676390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500855ab-0308-48f2-b575-6ed75dfa442e service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:15.922032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:15.922032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:15.922032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:15.922032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] PATH_INFO: `/v3/users/fe9a377646704edb8e389c490bbd0512` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:15.922620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9a377646704edb8e389c490bbd0512)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:15.922620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:15.922620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:16.879607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9a377646704edb8e389c490bbd0512'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:16.881455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9a377646704edb8e389c490bbd0512'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:16.894445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9a377646704edb8e389c490bbd0512'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:16.914117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9a377646704edb8e389c490bbd0512'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:17.058773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9a377646704edb8e389c490bbd0512'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:17.068075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1113/2245] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:15 2022] DELETE /identity/v3/users/fe9a377646704edb8e389c490bbd0512 => generated 0 bytes in 1562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:17.075740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:17.076343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:17.076343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:17.098619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:17.100297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:17.192574 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1133/2246] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:17.199858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:17.201635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:17.288438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12651a-5391-4fc7-b7b6-dc113eab699f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:17.293054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:17.293674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:17.293952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:17.294226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:17.308761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:17.310992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:17.324308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1114/2247] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:17.832783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:17.832783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:17.851912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96d7b-9529-40c7-8d69-440fc35f33e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:17.858816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:17.859901 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:17.860284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:17.860660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] PATH_INFO: `/v3/projects/c0cd1eea732046b2bf7e8764853ad6d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:17.862660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0cd1eea732046b2bf7e8764853ad6d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:17.865238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:17.963787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0cd1eea732046b2bf7e8764853ad6d8', 'request_id': 'req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:17.976204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0cd1eea732046b2bf7e8764853ad6d8', 'request_id': 'req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.055725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0cd1eea732046b2bf7e8764853ad6d8', 'request_id': 'req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.074486 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] The token cache is being invalidate because project c0cd1eea732046b2bf7e8764853ad6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:18.074801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.076909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.080202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1134/2248] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:17 2022] DELETE /identity/v3/projects/c0cd1eea732046b2bf7e8764853ad6d8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:18.084637 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:18.105347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fe3fe-3a70-4d25-aa8a-218ab6b9296c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:18.184072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:18.185226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:18.185567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:18.186004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] PATH_INFO: `/v3/users/cfbe0287e6b64e59be1263f395e3cf49` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:18.187284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbe0287e6b64e59be1263f395e3cf49)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:18.190269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:18.469762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:18.486712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbe0287e6b64e59be1263f395e3cf49'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.487328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbe0287e6b64e59be1263f395e3cf49'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.501746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbe0287e6b64e59be1263f395e3cf49'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.519210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbe0287e6b64e59be1263f395e3cf49'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.617218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbe0287e6b64e59be1263f395e3cf49'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:18.631919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1115/2249] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:18 2022] DELETE /identity/v3/users/cfbe0287e6b64e59be1263f395e3cf49 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:19.344093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:19.346379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:19.362724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4b26f-09f2-4ac0-87b0-9fd5f6a42c59 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:19.470364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:19.471043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:19.471371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:19.471656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] PATH_INFO: `/v3/projects/143f6bdf3eb8435e978e877fc54a2c90` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:19.472840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] RBAC: Authorizing `identity:delete_project(project_id=143f6bdf3eb8435e978e877fc54a2c90)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:19.474751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:19.500611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:19.651696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143f6bdf3eb8435e978e877fc54a2c90', 'request_id': 'req-7d18a495-8c07-4a2c-be9d-a7418dc8286d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:19.683658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143f6bdf3eb8435e978e877fc54a2c90', 'request_id': 'req-7d18a495-8c07-4a2c-be9d-a7418dc8286d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:19.794402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143f6bdf3eb8435e978e877fc54a2c90', 'request_id': 'req-7d18a495-8c07-4a2c-be9d-a7418dc8286d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:19.817332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] The token cache is being invalidate because project 143f6bdf3eb8435e978e877fc54a2c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:19.817984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:19.827344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:19.840173 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1135/2250] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:19 2022] DELETE /identity/v3/projects/143f6bdf3eb8435e978e877fc54a2c90 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:19.844888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:19.847277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:19.870867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6aed3d-4f05-4654-92e0-32a33f8eb43b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:19.973807 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:19.975946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:19.977825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:19.977825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] PATH_INFO: `/v3/users/6dbd1e37ab14459d851f4260a5b450f0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:19.979618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbd1e37ab14459d851f4260a5b450f0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:19.982187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:20.264261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbd1e37ab14459d851f4260a5b450f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:20.268488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbd1e37ab14459d851f4260a5b450f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:20.273851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbd1e37ab14459d851f4260a5b450f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:20.288601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbd1e37ab14459d851f4260a5b450f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:20.838190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5a78338-276e-4402-8495-a4bdf1692ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbd1e37ab14459d851f4260a5b450f0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:20.844683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1116/2251] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:19 2022] DELETE /identity/v3/users/6dbd1e37ab14459d851f4260a5b450f0 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:21.486220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:21.488225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:21.515461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a495-8c07-4a2c-be9d-a7418dc8286d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:21.662416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:21.663337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:21.663798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:21.664247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] PATH_INFO: `/v3/projects/89e05f16b68d48158ef0a6b497cc97d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:21.665644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e05f16b68d48158ef0a6b497cc97d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:21.668276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:21.839381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e05f16b68d48158ef0a6b497cc97d8', 'request_id': 'req-41fdb103-958b-477b-a350-39347e4b22d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:21.848126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e05f16b68d48158ef0a6b497cc97d8', 'request_id': 'req-41fdb103-958b-477b-a350-39347e4b22d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:21.960303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e05f16b68d48158ef0a6b497cc97d8', 'request_id': 'req-41fdb103-958b-477b-a350-39347e4b22d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:21.981350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] The token cache is being invalidate because project 89e05f16b68d48158ef0a6b497cc97d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:21.981656 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:21.982351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41fdb103-958b-477b-a350-39347e4b22d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:21.984821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1136/2252] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:21 2022] DELETE /identity/v3/projects/89e05f16b68d48158ef0a6b497cc97d8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:21.990428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:21.995502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:21.995502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.015047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:22.016684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:22.058208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:22.090719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1117/2253] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:22.099097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.106052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.106052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.122561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:22.123825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:22.136300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1137/2254] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:22.139389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.154553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e97ac-3d33-4d37-a4f0-752141d83ac9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.159723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.163207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.163207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.163207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.163207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:06:22.163207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.163685 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.179814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1118/2255] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:06:22.182104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.197244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89eb89-c8be-4ccc-bacd-53755311ebdc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.202294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.203488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.203903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.204246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.204936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.207013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.322438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1138/2256] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:22 2022] POST /identity/v3/projects => generated 409 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:06:22.323324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.333621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889761c-02d8-4eeb-8995-3f0388702256 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.414857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.415469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.415799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.416092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.416714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.418751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.512671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1119/2257] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:22 2022] POST /identity/v3/users => generated 305 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:06:22.517015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.525032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:22.531893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd6a2a-1d54-44c4-a4c1-fe30b9c03ad5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.542439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.543171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.549300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.549300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.549300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.549300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.570080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1139/2258] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:22.574063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.594385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b680f-0a00-40d6-9ce1-5e629536053b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.602350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.603643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.604397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.605031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.607344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.609760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8043b-b7e3-4098-91d0-bba7ffe0bf99 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.628748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1120/2259] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:22.632728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:22.647703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e366b85-51d0-460d-9e27-77510f3d419b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:22.656540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:22.657418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.657872 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.658300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] PATH_INFO: `/v3/projects/b6b828f2f4a447b28bac77adc1c10819/users/9be12196a48d47559676b8c781782315/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.692520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b828f2f4a447b28bac77adc1c10819, user_id=9be12196a48d47559676b8c781782315, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:22.695461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:22.854406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1140/2260] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:22 2022] PUT /identity/v3/projects/b6b828f2f4a447b28bac77adc1c10819/users/9be12196a48d47559676b8c781782315/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:22.866800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:22.867185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:22.867568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:22.941954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] MFA Rules not processed for user `9be12196a48d47559676b8c781782315`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:22.943479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:23.036599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1121/2261] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:23.037316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:23.176190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cd635-34be-4068-89c9-8ea1c6cfdafa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:23.182064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:23.182740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:23.183028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:23.183412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:23.184100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:23.186261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:23.310871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1141/2262] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:23 2022] POST /identity/v3/projects => generated 409 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:23.312677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:23.331169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0645b1-f15e-4497-9b72-7116e175a1f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:23.438834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:23.439643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:23.439967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:23.440244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:23.440888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:23.443016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:23.542881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1122/2263] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:23 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:23.547465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:23.559335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a1517-2db8-4157-9a2c-e8622b0ea0d4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:23.564765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:23.565380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:23.565723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:23.565996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:23.566643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:23.568279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:23.579900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1142/2264] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:23.584222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:23.610191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da339b7-c007-47f8-b59b-46a30b0b8eb3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:23.616866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:23.619987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:23.619987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:23.619987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] PATH_INFO: `/v3/projects/5151b5f4cafb4cff84445e879ff28e4d/users/5c50e5b58fc0445b937ad9bf8b880dc8/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:23.646273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5151b5f4cafb4cff84445e879ff28e4d, user_id=5c50e5b58fc0445b937ad9bf8b880dc8, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:23.649394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:23.767348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1123/2265] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:23 2022] PUT /identity/v3/projects/5151b5f4cafb4cff84445e879ff28e4d/users/5c50e5b58fc0445b937ad9bf8b880dc8/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:23.768765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:23.787820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a55a-370a-4ad0-8cc5-fde56b482fce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:23.883864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:23.920698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:23.921352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:23.921634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:23.921904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:23.928680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:23.930516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd7475-67a8-471f-ac25-bcd636882f92 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:23.989816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1143/2266] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:23.989938 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:24.008777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072a29-a3dd-4801-bdce-544aed82eed5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:24.016117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:24.017519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:24.017905 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:24.018486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] PATH_INFO: `/v3/projects/5151b5f4cafb4cff84445e879ff28e4d/users/5c50e5b58fc0445b937ad9bf8b880dc8/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:24.020925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5151b5f4cafb4cff84445e879ff28e4d, user_id=5c50e5b58fc0445b937ad9bf8b880dc8, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:24.024136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:24.060868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1124/2267] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:23 2022] PUT /identity/v3/projects/5151b5f4cafb4cff84445e879ff28e4d/users/5c50e5b58fc0445b937ad9bf8b880dc8/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:24.061457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d79b-9eb8-4851-852f-5b71cd75e3cc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:24.061457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d79b-9eb8-4851-852f-5b71cd75e3cc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:24.061457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d79b-9eb8-4851-852f-5b71cd75e3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:24.094516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fa04d79b-9eb8-4851-852f-5b71cd75e3cc None None] MFA Rules not processed for user `5c50e5b58fc0445b937ad9bf8b880dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:24.095820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fa04d79b-9eb8-4851-852f-5b71cd75e3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:24.149961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:24.199557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1144/2268] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:24.212623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:24.214308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:24.280575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143b088-7871-4150-85fc-7a7326efdb85 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:24.284303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:24.284918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:24.285226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:24.285500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:24.295294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:24.296783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:24.306983 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1125/2269] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:24.695970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:24.696511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:24.696609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:24.716737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:24.718856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:24.815674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1145/2270] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:24.898607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:24.901198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:25.012578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65481-3574-4306-9414-7d40a42861d2 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:25.018958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:25.020370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:25.020651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:25.020925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:25.031146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:25.039497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:25.042788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1126/2271] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:06:25.135214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:25.156148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dd67-1416-4ce4-aef4-8fe9b0cbf8c6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:25.163151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:25.163939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:25.164222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:25.164497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:25.260905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:25.262874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688c6fa-47fc-4a71-ac75-02cb735a0616 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:25.274765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1146/2272] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:25 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:35.226174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:35.913995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=70 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:39.562324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:39.632097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d801cfe-477e-4648-a91c-ebae638e9852 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:39.637502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:39.638288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:39.639191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:39.639600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] PATH_INFO: `/v3/users/9be12196a48d47559676b8c781782315` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:39.640906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be12196a48d47559676b8c781782315)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:39.643256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:39.910255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be12196a48d47559676b8c781782315'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:39.911117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be12196a48d47559676b8c781782315'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:39.918346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be12196a48d47559676b8c781782315'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:39.943547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be12196a48d47559676b8c781782315'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:40.002758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be12196a48d47559676b8c781782315'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:40.009912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1127/2273] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:39 2022] DELETE /identity/v3/users/9be12196a48d47559676b8c781782315 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:06:40.015618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:40.033427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:40.033427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:40.041873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:40.043540 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:40.117978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1147/2274] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:40.128738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:40.130221 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:40.193825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1f54-a85a-4686-940c-64734252a92c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:40.198969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:40.199816 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:40.200093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:40.200370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:40.212606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:40.214052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:40.224598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1128/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:40.843114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:40.844726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:40.857203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008b67e-471e-483e-8c15-e64b8124b92c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:40.861969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:40.862607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:40.862886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:40.863162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] PATH_INFO: `/v3/projects/b6b828f2f4a447b28bac77adc1c10819` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:40.864465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b828f2f4a447b28bac77adc1c10819)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:40.866231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:40.995667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b828f2f4a447b28bac77adc1c10819', 'request_id': 'req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.005029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b828f2f4a447b28bac77adc1c10819', 'request_id': 'req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.099859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b828f2f4a447b28bac77adc1c10819', 'request_id': 'req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.113706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] The token cache is being invalidate because project b6b828f2f4a447b28bac77adc1c10819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:41.116930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.116930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.116930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1148/2276] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:40 2022] DELETE /identity/v3/projects/b6b828f2f4a447b28bac77adc1c10819 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:41.120176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:41.131941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235552fe-ab3c-47d9-94e7-2ec417d3c484 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:41.155469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:41.203891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:41.204560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:41.204841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:41.205118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] PATH_INFO: `/v3/users/5c50e5b58fc0445b937ad9bf8b880dc8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:41.206398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c50e5b58fc0445b937ad9bf8b880dc8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:41.208322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:41.396644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c50e5b58fc0445b937ad9bf8b880dc8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.397272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c50e5b58fc0445b937ad9bf8b880dc8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.404895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c50e5b58fc0445b937ad9bf8b880dc8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.416584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c50e5b58fc0445b937ad9bf8b880dc8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.486951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a5234d5c-312b-4d40-8e4a-689fc7a7a818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c50e5b58fc0445b937ad9bf8b880dc8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:41.493108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1129/2277] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:41 2022] DELETE /identity/v3/users/5c50e5b58fc0445b937ad9bf8b880dc8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:42.129445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:42.134048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:42.155405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f36cb-adad-4b0e-bf83-0ca4075f375d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:42.228469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:42.229634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.229916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.230186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] PATH_INFO: `/v3/projects/5151b5f4cafb4cff84445e879ff28e4d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.231393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5151b5f4cafb4cff84445e879ff28e4d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:42.233254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:42.404709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:42.422383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5151b5f4cafb4cff84445e879ff28e4d', 'request_id': 'req-3c48a914-f057-4044-96d5-877a363f69d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:42.430398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5151b5f4cafb4cff84445e879ff28e4d', 'request_id': 'req-3c48a914-f057-4044-96d5-877a363f69d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:42.544545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5151b5f4cafb4cff84445e879ff28e4d', 'request_id': 'req-3c48a914-f057-4044-96d5-877a363f69d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:42.558536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] The token cache is being invalidate because project 5151b5f4cafb4cff84445e879ff28e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:42.559104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:42.560025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3c48a914-f057-4044-96d5-877a363f69d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:42.562665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1149/2278] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:42 2022] DELETE /identity/v3/projects/5151b5f4cafb4cff84445e879ff28e4d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:06:42.567579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.568114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.568438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.593502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:42.595075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:42.680001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1130/2279] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:42.688121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.688413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.688755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.709404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:42.710811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:42.719179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1150/2280] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:42.723347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:42.733562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb6696-1a27-43cb-9c7f-0fd7789dcd68 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:42.738467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:42.739008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.739286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.739554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.740225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:06:42.740544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:42.742123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:42.756259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1131/2281] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:06:42.760738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:42.770107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaa1e1-41e5-4643-a074-a9d3c403f267 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:42.775950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:42.776550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.776824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.777094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.777746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:42.780610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:42.896719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1151/2282] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:42 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:42.903460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:42.916731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2579a-4f0a-4fc4-a705-66a292bc6fb2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:42.948789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:42.983726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:42.984044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:42.984320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:42.984588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:42.985223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:42.987240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.163225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1132/2283] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:42 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:06:43.167477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:43.178317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef45be-a8a8-4d2e-97c1-aca2b5bc93c3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:43.183061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:43.183718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.183973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.184405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.185713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:43.187490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.202726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1152/2284] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:06:43.205588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:43.221131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd6982-1145-41ca-88fe-ee679ac16149 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:43.226422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:43.227215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.227503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.230555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.230555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:43.230555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c5a1-d189-45ef-a8b8-7398580fd424 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.240583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1133/2285] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:06:43.244873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:43.256400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d38f3-6878-4bba-b774-5bbd00ec51fe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:43.264915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:43.265180 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.265462 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.265740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] PATH_INFO: `/v3/projects/468ef231d69e45baa0cbd0b6a8d1120b/users/0d58f75a93054acbb196f9f53c8d444f/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.281793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468ef231d69e45baa0cbd0b6a8d1120b, user_id=0d58f75a93054acbb196f9f53c8d444f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:43.284509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.380593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1153/2286] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:43 2022] PUT /identity/v3/projects/468ef231d69e45baa0cbd0b6a8d1120b/users/0d58f75a93054acbb196f9f53c8d444f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:43.390883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.391183 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.391435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.422105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] MFA Rules not processed for user `0d58f75a93054acbb196f9f53c8d444f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:43.423314 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:43.506316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1134/2287] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:43.517694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:43.519360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:43.578499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab271c1-f2fa-4330-bffe-931be4aef5a1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:43.582759 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:43.583376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.583652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.583947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.593633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:43.594954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10082b31-ca67-4973-a8ce-77d1895a16ca service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.606130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1154/2288] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:43.626289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:43.688750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d953e-d89b-4f79-8dcc-c600053dde7d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:43.693662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:43.694282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.694559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.694834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] PATH_INFO: `/v3/users/0d58f75a93054acbb196f9f53c8d444f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.696066 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d58f75a93054acbb196f9f53c8d444f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:43.698092 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:43.862970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:43.869153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d58f75a93054acbb196f9f53c8d444f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:43.870050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d58f75a93054acbb196f9f53c8d444f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:43.877993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d58f75a93054acbb196f9f53c8d444f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:43.890365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d58f75a93054acbb196f9f53c8d444f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:43.944364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d58f75a93054acbb196f9f53c8d444f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:43.950297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1135/2289] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:43 2022] DELETE /identity/v3/users/0d58f75a93054acbb196f9f53c8d444f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:43.955529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:43.955860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:43.956172 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:43.975536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:43.976778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:44.023532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:44.035421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1155/2290] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:44.046479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:44.047953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:44.112768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a304-6bbd-419a-a277-0132ddec7746 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:44.116940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:44.117547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:44.117824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:44.118096 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:44.129162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:44.130788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df516057-4a14-44a3-a8a8-58e78b6e8005 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:44.142800 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1136/2291] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:44.643658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:44.646285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:44.658724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e04992-eb86-440a-954c-99c7cd9e5aa8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:44.664052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:44.664726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:44.665008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:44.665313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] PATH_INFO: `/v3/projects/468ef231d69e45baa0cbd0b6a8d1120b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:44.667802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468ef231d69e45baa0cbd0b6a8d1120b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:44.668644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:44.779343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468ef231d69e45baa0cbd0b6a8d1120b', 'request_id': 'req-089f0186-0e8a-422e-898c-8c887c9be8b3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:44.786830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468ef231d69e45baa0cbd0b6a8d1120b', 'request_id': 'req-089f0186-0e8a-422e-898c-8c887c9be8b3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:44.848773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468ef231d69e45baa0cbd0b6a8d1120b', 'request_id': 'req-089f0186-0e8a-422e-898c-8c887c9be8b3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:44.860410 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] The token cache is being invalidate because project 468ef231d69e45baa0cbd0b6a8d1120b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:44.860707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:44.861517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-089f0186-0e8a-422e-898c-8c887c9be8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:44.863104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1156/2292] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:44 2022] DELETE /identity/v3/projects/468ef231d69e45baa0cbd0b6a8d1120b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:44.868015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:44.868596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:44.868596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:44.889505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:44.890624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:44.963810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1137/2293] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:44.972627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:44.973503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:44.973800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:44.999673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:45.001633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:45.018565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1157/2294] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:45.023504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.037143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8129-e352-487e-8d4c-169ce0c95ab4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.042207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.042892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.043297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.043635 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.044925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:06:45.045228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.046939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.059698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1138/2295] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:06:45.064680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.074279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c524bb0-1193-4658-b5dc-a34399830439 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.079300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.079940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.080250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.080530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.081162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.083083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.186499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1158/2296] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:45 2022] POST /identity/v3/projects => generated 361 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:45.190895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.200790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964d7c-a209-4ec7-b33b-2c2e005151b8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.244857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:45.259424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.260068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.260622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.260622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.264756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.264756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.432110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1139/2297] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:45 2022] POST /identity/v3/users => generated 281 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:45.441234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.452384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc09731-66af-45a3-bcdf-f0777ad82bb6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.456936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.457534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.457806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.458073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.458706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.460315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.470762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1159/2298] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:06:45.475958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.486379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248570-b9c0-4d77-b5ff-5e4fd9eb605b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.491662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.492406 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.495601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.495601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.495601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.495601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ad615-14fc-4f2b-b1a4-91e27b52eec2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.510964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1140/2299] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:45.516266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.526372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104f37f-6e3a-4605-8fd3-102774cdd7b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.531030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.531671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.531984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.532255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] PATH_INFO: `/v3/projects/03dc43991f4a41e495f4e1a56499755f/users/9438e6ad7a41499e89346eccf6a2affc/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.548048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dc43991f4a41e495f4e1a56499755f, user_id=9438e6ad7a41499e89346eccf6a2affc, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.550754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.637195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1160/2300] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:45 2022] PUT /identity/v3/projects/03dc43991f4a41e495f4e1a56499755f/users/9438e6ad7a41499e89346eccf6a2affc/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:45.647625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.647963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.648239 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.681516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] MFA Rules not processed for user `9438e6ad7a41499e89346eccf6a2affc`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:45.682933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:45.756904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1141/2301] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:45.791915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.793733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:45.855036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd452f-5641-43db-a003-788a7e56e4fd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.859256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.859817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.860100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.860370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.870608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.874693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a3d9c-7500-4b68-8822-2d4a78198215 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:45.882250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1161/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:45.898014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:45.967141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d5755-a775-4af9-90ea-0ca767a061c7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:45.971665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:45.972321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:45.972603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:45.972856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] PATH_INFO: `/v3/users/9438e6ad7a41499e89346eccf6a2affc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:45.974005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9438e6ad7a41499e89346eccf6a2affc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:45.975977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:46.253151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9438e6ad7a41499e89346eccf6a2affc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:46.253808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9438e6ad7a41499e89346eccf6a2affc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:46.261068 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9438e6ad7a41499e89346eccf6a2affc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:46.262223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:46.273840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9438e6ad7a41499e89346eccf6a2affc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:46.356299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9438e6ad7a41499e89346eccf6a2affc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:46.362708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1142/2303] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:06:45 2022] DELETE /identity/v3/users/9438e6ad7a41499e89346eccf6a2affc => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:46.367200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:46.367502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:46.367792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:46.386754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:46.392113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:46.399362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:46.450657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1162/2304] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:46.461323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:46.462850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:46.533906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e469-2619-4611-b0eb-d10796a52b3b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:46.538407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:46.538764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:46.539074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:46.539847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:46.549629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:46.551054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b86027-7504-472a-9de3-310975e67233 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:46.561616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1143/2305] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:47.034750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.034750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:47.047970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e883b-7e66-4c85-94c5-b801eeaeddcd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.053696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.054521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.054958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.055388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] PATH_INFO: `/v3/projects/03dc43991f4a41e495f4e1a56499755f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.062313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=03dc43991f4a41e495f4e1a56499755f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.062313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.185773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03dc43991f4a41e495f4e1a56499755f', 'request_id': 'req-1799f078-0e38-47ed-afad-1a64576b74bb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:47.192216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03dc43991f4a41e495f4e1a56499755f', 'request_id': 'req-1799f078-0e38-47ed-afad-1a64576b74bb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:47.293175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03dc43991f4a41e495f4e1a56499755f', 'request_id': 'req-1799f078-0e38-47ed-afad-1a64576b74bb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:47.305282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] The token cache is being invalidate because project 03dc43991f4a41e495f4e1a56499755f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:06:47.305571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:47.306218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1799f078-0e38-47ed-afad-1a64576b74bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:06:47.307711 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1163/2306] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:06:47 2022] DELETE /identity/v3/projects/03dc43991f4a41e495f4e1a56499755f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:47.313550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.314103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.314103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.335933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:47.337380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:47.405063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1144/2307] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:06:47.413836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.414129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.414432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.437749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:47.438936 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:47.446837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1164/2308] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:47.451062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.462002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e28745-2cdc-4fbc-86e5-ea4ee965798d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.467138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.467731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.468035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.468301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.468939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:06:47.469238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.470812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.483205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1145/2309] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:06:47.487911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.498195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0115bd5-e9b7-4d82-aaf5-704f981b386a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.502882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.503475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.503750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.504055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.504699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.506634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.621582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1165/2310] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:06:47 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:06:47.625947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.635811 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a10247-3b10-4f98-b9db-d5519f391bc5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.690728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.691323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.691601 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.691918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.692527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.694417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.703993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:47.809618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1146/2311] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:06:47 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:06:47.818463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.829734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6ba4c-d122-4b8c-a0d8-b80bf80e10e6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.834807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.835440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.835713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.836014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.836827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.838409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.850065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1166/2312] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:47.853978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.864562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25630019-4047-467b-ba43-51f2d824dee0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.869809 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec86fd-3c80-4d16-a7b5-8a3ef52f17e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:47.884205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1147/2313] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:06:47.889282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:47.900468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792c731-60f5-465d-891d-978d36b489bc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:47.905971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:47.907643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:47.907643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:47.907643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] PATH_INFO: `/v3/projects/12b69ab4970948c2acd12e1dc5e57363/users/504c2804020d47e3bd592dc06a0ba9db/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:47.925140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b69ab4970948c2acd12e1dc5e57363, user_id=504c2804020d47e3bd592dc06a0ba9db, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:47.928625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:48.110327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1167/2314] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:06:47 2022] PUT /identity/v3/projects/12b69ab4970948c2acd12e1dc5e57363/users/504c2804020d47e3bd592dc06a0ba9db/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:06:48.121599 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9c89-d3fa-466b-91bb-299a8ee9d839 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:48.121822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9c89-d3fa-466b-91bb-299a8ee9d839 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:48.122064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9c89-d3fa-466b-91bb-299a8ee9d839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:48.154451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-59cd9c89-d3fa-466b-91bb-299a8ee9d839 None None] MFA Rules not processed for user `504c2804020d47e3bd592dc06a0ba9db`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:48.155832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-59cd9c89-d3fa-466b-91bb-299a8ee9d839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:48.230326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1148/2315] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:06:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:48.242752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:48.244917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:48.287811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:48.305498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9174f-b459-42f8-954c-55004eb439e5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:48.309925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:48.310546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:48.310820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:48.311088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:48.319860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:48.321573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d15c9-f0a0-4861-b01f-39d8dbf69467 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:48.333447 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1168/2316] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:50.846225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0fcb7-948f-4ab5-9d19-03f6bcab29a2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:50.848236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0fcb7-948f-4ab5-9d19-03f6bcab29a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:50.848236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0fcb7-948f-4ab5-9d19-03f6bcab29a2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:50.848733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1149/2317] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:06:50.855605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ce15-d32b-4c69-b4d3-0e5be2c10cea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:50.856956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ce15-d32b-4c69-b4d3-0e5be2c10cea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:50.856956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ce15-d32b-4c69-b4d3-0e5be2c10cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:50.884362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6304ce15-d32b-4c69-b4d3-0e5be2c10cea None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:50.885683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6304ce15-d32b-4c69-b4d3-0e5be2c10cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:50.993613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1169/2318] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:51.007690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0fcb7-948f-4ab5-9d19-03f6bcab29a2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:51.019279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0fcb7-948f-4ab5-9d19-03f6bcab29a2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:51.022844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:51.023443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.023721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.024024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.036274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:51.037870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c2cdc-55c6-4c5a-8cc0-85856cf54099 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:51.050770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1150/2319] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:51.404031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.404031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.404395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.405597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1170/2320] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:06:51.410636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa597-d6cb-4f1f-8b53-a97fc4ceb99e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.411204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa597-d6cb-4f1f-8b53-a97fc4ceb99e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.411554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa597-d6cb-4f1f-8b53-a97fc4ceb99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.433082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1b1aa597-d6cb-4f1f-8b53-a97fc4ceb99e None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:51.434919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b1aa597-d6cb-4f1f-8b53-a97fc4ceb99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:51.501611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1151/2321] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:51.516316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:51.519871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:51.534612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec032e9-d64c-409d-9824-7d8283f06747 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:51.538508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:51.539114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.539451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.539725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.548969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:51.550407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e36ea-6c1f-49ef-b15c-2c280108b382 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:51.558737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1171/2322] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:51.658246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.658462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.658715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.660135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1152/2323] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:06:51.666385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.666700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.666981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.690596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:51.691951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:51.702302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1172/2324] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:51.712969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:51.715212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:51.730037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5fea0-ed33-4cee-a8e7-c17b7394cea3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:51.735492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:51.735722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:51.735969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:51.736248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:51.749247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:51.750946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:51.764881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1153/2325] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:53.024348 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:53.025967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:53.036774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:53.038214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:53.165752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:53.173832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec182c3-de5e-4674-8243-667c6ad4972c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:53.177483 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fd694-2da1-4bd6-82a1-9ce512f2f58e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:53.180741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:53.181380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:53.181713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:53.182092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:53.182896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:53.183731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:53.184073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:53.184350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:53.184466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:53.187560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:53.198873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:53.201080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22189bf2-e1db-4c83-a1f9-07cc04315728 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:53.210402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:53.210402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:53.219685 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1173/2326] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:06:53.225009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1154/2327] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:06:55.112337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419802fd-349f-4ac3-a819-0f7d643df927 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:55.112934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419802fd-349f-4ac3-a819-0f7d643df927 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:55.113249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419802fd-349f-4ac3-a819-0f7d643df927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:55.135535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-419802fd-349f-4ac3-a819-0f7d643df927 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:55.136780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-419802fd-349f-4ac3-a819-0f7d643df927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:55.180344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:06:55.206431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1174/2328] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:55.224686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:55.226158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:55.299620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a9502-f423-483e-9309-2416d7a828ad service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:55.306794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:55.309733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:55.309733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:55.309733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:55.323249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:55.325858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:55.335389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1155/2329] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:06:56.401699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b077a-7f1b-45a3-961d-38b5b8b65b7e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:56.402235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b077a-7f1b-45a3-961d-38b5b8b65b7e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:56.402835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b077a-7f1b-45a3-961d-38b5b8b65b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:56.423226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-036b077a-7f1b-45a3-961d-38b5b8b65b7e None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:56.424457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-036b077a-7f1b-45a3-961d-38b5b8b65b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:56.432090 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1175/2330] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:56.445319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:56.455610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e7b8-aa7a-46e6-9e96-df8b65e7ca45 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:56.460619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:56.461212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:56.461490 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:56.461760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:56.470300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:56.471886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:56.479707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1156/2331] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:06:57.261802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939911a3-6684-40b7-af8f-5a3ec2127469 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:57.262177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939911a3-6684-40b7-af8f-5a3ec2127469 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:57.262534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939911a3-6684-40b7-af8f-5a3ec2127469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:57.283884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-939911a3-6684-40b7-af8f-5a3ec2127469 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:57.285380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-939911a3-6684-40b7-af8f-5a3ec2127469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:57.294526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1176/2332] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:57.318859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:57.332271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09815d99-b306-4087-b95f-97684e2cbe2e service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:57.338904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:57.339519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:57.340350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:57.340621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:57.350070 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:57.351937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:57.360549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1157/2333] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:06:57.942852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884db5-5b57-4192-99d8-4dc5f7196c57 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:57.943361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884db5-5b57-4192-99d8-4dc5f7196c57 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:57.943457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1884db5-5b57-4192-99d8-4dc5f7196c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:57.962873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c1884db5-5b57-4192-99d8-4dc5f7196c57 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:06:57.964204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c1884db5-5b57-4192-99d8-4dc5f7196c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:06:57.971273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1177/2334] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:06:57.983739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:06:57.994211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b437d58-bfa6-42c3-8d1a-04c989930099 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:06:57.999921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:06:58.000504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:06:58.000780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:06:58.001054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:06:58.008854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:06:58.010467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:06:58.018883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1158/2335] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:07:03.348584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d65da-a446-4a86-bae7-821d97c4c2c9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:03.354076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d65da-a446-4a86-bae7-821d97c4c2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:03.354786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d65da-a446-4a86-bae7-821d97c4c2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:03.379827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d47d65da-a446-4a86-bae7-821d97c4c2c9 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:03.381021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d47d65da-a446-4a86-bae7-821d97c4c2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:03.388037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1178/2336] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:03.400898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:03.412230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd7e09-6989-4c2a-b987-64dc3c147291 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:03.417212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:03.418361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:03.418641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:03.418917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:03.428828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:03.430748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:03.439354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1159/2337] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:07:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 11:07:05.138852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09578b-5c35-41b4-8f0d-e94d7eaf8285 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:05.139437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09578b-5c35-41b4-8f0d-e94d7eaf8285 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:05.139437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09578b-5c35-41b4-8f0d-e94d7eaf8285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:05.159847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8f09578b-5c35-41b4-8f0d-e94d7eaf8285 None None] MFA Rules not processed for user `f2e7493d38d24493b3298ac37592ecd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:05.161433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8f09578b-5c35-41b4-8f0d-e94d7eaf8285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:05.218860 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1179/2338] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:05.251665 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:05.253120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:05.358024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f55bd-f971-416c-8ed9-b31f639df5e1 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:05.369769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:05.375056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:05.375056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:05.375056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] PATH_INFO: `/v3/users/c727b8f265774a4bb03d8425212e7540` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:05.375056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c727b8f265774a4bb03d8425212e7540)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:05.375056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:05.421282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:05.702982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c727b8f265774a4bb03d8425212e7540'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:05.703746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c727b8f265774a4bb03d8425212e7540'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:05.712207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c727b8f265774a4bb03d8425212e7540'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:05.742133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c727b8f265774a4bb03d8425212e7540'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:05.880706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c727b8f265774a4bb03d8425212e7540'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:05.895100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1160/2339] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:07:05 2022] DELETE /identity/v3/users/c727b8f265774a4bb03d8425212e7540 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:05.900075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:05.906910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:05.906910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:05.950259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:05.951897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:06.103411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1180/2340] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:06.113566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:06.116786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:06.213677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2ea1-365e-4258-93ca-f584dc237ea0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:06.221488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:06.222286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:06.235698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:06.236276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:06.247888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:06.249553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:06.263897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1161/2341] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:07:06.585250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:06.586748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:06.618601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a01a4-c000-4537-84b6-6cbc512a60e7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:06.633103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:06.633788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:06.634070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:06.634343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] PATH_INFO: `/v3/projects/83857835f34141f18a89c759631878f6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:06.635549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83857835f34141f18a89c759631878f6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:06.637719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:06.775624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83857835f34141f18a89c759631878f6', 'request_id': 'req-88d65a01-fbfe-4a66-82f4-3715e7b755c7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:06.783929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83857835f34141f18a89c759631878f6', 'request_id': 'req-88d65a01-fbfe-4a66-82f4-3715e7b755c7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:06.788481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:06.908780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83857835f34141f18a89c759631878f6', 'request_id': 'req-88d65a01-fbfe-4a66-82f4-3715e7b755c7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:06.920354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] The token cache is being invalidate because project 83857835f34141f18a89c759631878f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:07:06.920644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:06.921344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:06.922862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1181/2342] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:07:06 2022] DELETE /identity/v3/projects/83857835f34141f18a89c759631878f6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:16.261184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=46,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:17.079961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:26.353253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:26.354831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:26.367342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb0aac-d8fe-4bb8-a262-90b60e5e62ae service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:26.435626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:26.436486 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:26.436773 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:26.437053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:26.439463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:26.504912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:26.506638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3563927-bc1d-48fd-9a32-04ef16607e8f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:26.518138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1162/2343] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:07:29.261413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:29.263003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:29.359798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65a01-fbfe-4a66-82f4-3715e7b755c7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:29.392915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:29.392915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:29.392915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:29.392915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] PATH_INFO: `/v3/users/504c2804020d47e3bd592dc06a0ba9db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:29.393529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=504c2804020d47e3bd592dc06a0ba9db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:29.393529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:29.654194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504c2804020d47e3bd592dc06a0ba9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:29.654852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504c2804020d47e3bd592dc06a0ba9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:29.667593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504c2804020d47e3bd592dc06a0ba9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:29.746941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504c2804020d47e3bd592dc06a0ba9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:29.754545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504c2804020d47e3bd592dc06a0ba9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:29.761569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1182/2344] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:07:29 2022] DELETE /identity/v3/users/504c2804020d47e3bd592dc06a0ba9db => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:29.765723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:29.766193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:29.766291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:29.786837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:29.788095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:29.854074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1163/2345] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:29.861277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:29.865798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:29.935724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeec8b-d880-416e-9851-6d3b79f4c59e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:29.939608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:29.941724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:29.941724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:29.941724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:29.950904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:29.952455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a6204-a5f4-4522-b295-61a3542c00a2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:29.962261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1183/2346] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:07:30.515168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:30.517168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:30.543823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca0731-1b6f-44cb-b88e-bc0aa1cd39be None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:30.549535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:30.550613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:30.553492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:30.553492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] PATH_INFO: `/v3/projects/12b69ab4970948c2acd12e1dc5e57363` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:30.553797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b69ab4970948c2acd12e1dc5e57363)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:30.557264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:30.724270 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:30.725892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b69ab4970948c2acd12e1dc5e57363', 'request_id': 'req-07ff7ccb-812b-444d-bdc2-764e64ffa935'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:30.733061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b69ab4970948c2acd12e1dc5e57363', 'request_id': 'req-07ff7ccb-812b-444d-bdc2-764e64ffa935'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:30.838679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b69ab4970948c2acd12e1dc5e57363', 'request_id': 'req-07ff7ccb-812b-444d-bdc2-764e64ffa935'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:30.856243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] The token cache is being invalidate because project 12b69ab4970948c2acd12e1dc5e57363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:07:30.856863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:30.857705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-07ff7ccb-812b-444d-bdc2-764e64ffa935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:30.863525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1164/2347] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:07:30 2022] DELETE /identity/v3/projects/12b69ab4970948c2acd12e1dc5e57363 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:30.865514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:30.865994 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:30.866362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:30.889731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:30.890962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:30.911525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:30.954513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1184/2348] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:30.962505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:30.963289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:30.963568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:30.983783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:30.984997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:30.992972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1165/2349] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:30.996921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.006542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85318da-4a1b-4917-aacd-b0f52bf45582 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.011175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.017773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.026502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1185/2350] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:07:31.030928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.040692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ee2c-dd88-4673-a729-36ac4c6c6b83 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.046048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.046650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.046939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.047204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.047879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.051898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.183141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1166/2351] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:07:31 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:07:31.201062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.201062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c600b95-e8ac-4ca6-870a-8ab51f8fc856 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.257775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.258374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.258703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.258977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.260202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.262106 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.429941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1186/2352] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:07:31 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:07:31.431232 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.441542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad718f7-8bb0-4b51-bc7e-85ad9ff75ec0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.446302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.446897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.447674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.447981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.448727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.450392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.461364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1167/2353] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:07:31.465701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.479710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94280a-bef2-4f1b-9887-0dd9515cd1e5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.485611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.486622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.487113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.487597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.488807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.492196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ec21a-61ad-49ea-8ba1-68b86ee61450 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.504285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1187/2354] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:07:31.511397 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.521375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016ec5-eed8-427f-a48a-414b6b1b59c4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.526631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.527280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.527559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.527898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] PATH_INFO: `/v3/projects/e96532e2531e4f9f813a10fa894205d6/users/ee6c5f91e98d4f6ba3d468c3c79bc129/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.542502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96532e2531e4f9f813a10fa894205d6, user_id=ee6c5f91e98d4f6ba3d468c3c79bc129, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.545240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.644760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1168/2355] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:07:31 2022] PUT /identity/v3/projects/e96532e2531e4f9f813a10fa894205d6/users/ee6c5f91e98d4f6ba3d468c3c79bc129/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:31.656367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56761-8a84-4c4a-b21e-022736759db8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.656659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56761-8a84-4c4a-b21e-022736759db8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.656935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f56761-8a84-4c4a-b21e-022736759db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.685404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-29f56761-8a84-4c4a-b21e-022736759db8 None None] MFA Rules not processed for user `ee6c5f91e98d4f6ba3d468c3c79bc129`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:31.686616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29f56761-8a84-4c4a-b21e-022736759db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:31.778933 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1188/2356] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:07:31.790176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:31.792124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:31.867585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f387d86-864c-4bcf-97ad-d5c04aa25907 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:31.873295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:31.873918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:31.874198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:31.874475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:31.884644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:31.886581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:31.900322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1169/2357] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:07:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:07:39.458437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:39.458963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:39.458963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:39.483445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:39.485025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:39.498905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:39.556288 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1189/2358] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:39.570173 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:39.571612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:39.643028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffbbd1-30ec-4595-9671-9e6626dfb962 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:39.648418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:39.649079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:39.649338 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:39.649640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:39.659370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:39.661005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabf2cd-9446-40ec-9cd2-57cedbc336aa service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:39.671047 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1170/2359] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:07:43.130401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:43.202303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680419f6-f276-431e-b3d5-c98e139dbb5a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:43.206827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:43.209848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:43.209848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:43.209848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] PATH_INFO: `/v3/users/ee6c5f91e98d4f6ba3d468c3c79bc129` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:43.209848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6c5f91e98d4f6ba3d468c3c79bc129)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:43.211011 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:43.446305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6c5f91e98d4f6ba3d468c3c79bc129'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:43.446955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6c5f91e98d4f6ba3d468c3c79bc129'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:43.462183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6c5f91e98d4f6ba3d468c3c79bc129'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:43.534411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6c5f91e98d4f6ba3d468c3c79bc129'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:43.544888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6c5f91e98d4f6ba3d468c3c79bc129'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:43.555232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1190/2360] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:07:43 2022] DELETE /identity/v3/users/ee6c5f91e98d4f6ba3d468c3c79bc129 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:43.562129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:43.563586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:43.564385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:43.578499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:43.598304 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:07:43.600050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:43.678036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1171/2361] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:07:43.707361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:43.709742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:43.772412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:43.785882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dba55d-f736-4f6d-99f5-196ab02a78cf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:43.791879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:43.791879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:43.791879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:43.791879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:43.804254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:43.805883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04004c7f-7ba4-446a-991f-92bbe33f692c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:43.819076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1191/2362] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:07:44.598857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:07:44.600675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:07:44.612967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36aca9-98fa-4b11-bca3-66d521e769f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:07:44.618888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:07:44.618888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:07:44.618888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:07:44.618888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] PATH_INFO: `/v3/projects/e96532e2531e4f9f813a10fa894205d6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:07:44.620164 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96532e2531e4f9f813a10fa894205d6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:07:44.622107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:07:44.780985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96532e2531e4f9f813a10fa894205d6', 'request_id': 'req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:44.787410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96532e2531e4f9f813a10fa894205d6', 'request_id': 'req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:44.872177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96532e2531e4f9f813a10fa894205d6', 'request_id': 'req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:44.883402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] The token cache is being invalidate because project e96532e2531e4f9f813a10fa894205d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:07:44.883682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:44.884415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c1c99c4-684c-463c-8baf-eb5fa27f0c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:07:44.886554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1172/2363] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:07:44 2022] DELETE /identity/v3/projects/e96532e2531e4f9f813a10fa894205d6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:07:53.842431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:07:54.880762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:06.731681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:06.732215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:06.732368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:06.751199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] MFA Rules not processed for user `684d59041e82466d8aad6d2964842bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:06.752417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:06.805807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1192/2364] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:08:16.799261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=25 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:35.670231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:35.670725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:35.670836 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:35.693025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] MFA Rules not processed for user `1b7724f3ef4543ed9c3df0455ae4fb76`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:35.694110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:35.751874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1173/2365] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:08:37.553515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:37.554975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:37.613397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743ca66-7c3d-4499-9d96-5c903c8721ac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:37.618568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:37.619584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:37.619934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:37.620241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:37.622446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:37.697071 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:37.698515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:37.709794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1193/2366] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:08:38.827192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:38.836745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3c03b-0a8a-46b1-88f4-0a9d04728114 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:38.840351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:38.840951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:38.841230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:38.842059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:38.845089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:38.863081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:38.864505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:38.876097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1174/2367] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:08:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:08:43.937803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:43.939246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:44.006153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3a62-ea7d-46de-bdf2-3e6a50758442 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:44.010655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:44.011276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:44.011551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:44.011918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] PATH_INFO: `/v3/users/f2e7493d38d24493b3298ac37592ecd4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:44.013017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e7493d38d24493b3298ac37592ecd4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:44.014950 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:44.203343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:44.206567 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e7493d38d24493b3298ac37592ecd4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.211097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e7493d38d24493b3298ac37592ecd4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.218713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e7493d38d24493b3298ac37592ecd4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.277675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e7493d38d24493b3298ac37592ecd4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.284492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff22d432-8563-43ba-a2ea-1a08e73b38fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e7493d38d24493b3298ac37592ecd4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.291747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1194/2368] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:08:43 2022] DELETE /identity/v3/users/f2e7493d38d24493b3298ac37592ecd4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:08:44.542235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:44.553957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff7249-8544-4c3f-9108-4fc3feeb0177 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:44.613396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:44.614018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:44.614295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:44.614563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] PATH_INFO: `/v3/projects/23c8a1db00944c59950120c030ab47e3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:44.615657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c8a1db00944c59950120c030ab47e3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:44.617557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:44.767663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c8a1db00944c59950120c030ab47e3', 'request_id': 'req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.774071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c8a1db00944c59950120c030ab47e3', 'request_id': 'req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.900662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c8a1db00944c59950120c030ab47e3', 'request_id': 'req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.912587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] The token cache is being invalidate because project 23c8a1db00944c59950120c030ab47e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:08:44.912878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.913522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1744b6eb-4c5f-43ed-9810-de0278a3ce2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:08:44.915029 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1175/2369] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:08:44 2022] DELETE /identity/v3/projects/23c8a1db00944c59950120c030ab47e3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:08:44.920146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:44.920504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:44.921138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:44.942154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:44.943361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:45.004083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1195/2370] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:08:45.011612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.011942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.012220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.031219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:45.032436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:45.040720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1176/2371] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:08:45.044878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.055402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d338a6-f5ee-4a93-b773-2dcd9c3b2c91 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.060094 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.060690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.060964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.061231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.061883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:08:45.062184 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.063787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.076076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1196/2372] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:08:45.080116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.089326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca1c2-0621-4977-91ec-0db30ed11ce0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.093910 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.094507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.094779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.095044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.095670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.098132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.114400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:45.211628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1177/2373] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:08:45 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:08:45.215611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.226253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ba960-3ed6-4b96-9035-c683226e44d2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.280061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.280657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.280935 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.281205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.281819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.284401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.415543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1197/2374] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:08:45 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:08:45.419607 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.429897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449049c1-791f-4a6d-948d-a667894ed1b5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.434436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.434913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.435191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.435468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.436138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.437722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.447380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1178/2375] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:08:45.451134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.462014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c1725-4d78-44bb-95b0-c1e87baffaf3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.466775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.467365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.467640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.467964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.468597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.470242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324bf01-9ecb-4a62-a5a6-7131cec9015f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.481370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1198/2376] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:08:45.485520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.495293 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3ba0-9d0e-4c01-ab6a-dc7ee4abcf09 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.499729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.500377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.500656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.500928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] PATH_INFO: `/v3/projects/b1d394882f294c6283721bcc6f8ff04d/users/2a9dbe3b7fdd482681b6ab94c26f4e71/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.514997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d394882f294c6283721bcc6f8ff04d, user_id=2a9dbe3b7fdd482681b6ab94c26f4e71, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.517679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.609837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1179/2377] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:08:45 2022] PUT /identity/v3/projects/b1d394882f294c6283721bcc6f8ff04d/users/2a9dbe3b7fdd482681b6ab94c26f4e71/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:08:45.617733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.617733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.617733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.634961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:45.636281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:45.657786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:45.697446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1199/2378] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:08:45.707869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:45.709874 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:45.766477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48b0c2-3db8-4a7a-867b-2e4fdaaf87cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:45.770613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:45.771203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:45.771477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:45.771741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:45.780566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:45.781962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443144-3303-4923-a8d0-d6db07aceb95 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:45.793776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1180/2379] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:08:50.544279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:50.546077 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:50.635931 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72687b-8ca4-49b8-baf0-a5abfe38f8f7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:50.639793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:50.640445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:50.640722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:50.640993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:50.643500 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:50.714464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:50.715972 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:50.736006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1200/2380] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:08:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:08:51.467331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:51.468228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:51.468666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:51.544752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] MFA Rules not processed for user `2a9dbe3b7fdd482681b6ab94c26f4e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:08:51.550108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:51.666107 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1181/2381] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:08:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:08:51.670322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:51.684355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df381b12-14af-4bc9-b8fa-d31d395019e1 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:51.690756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:51.691373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:51.691746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:51.692126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:51.693072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:51.695230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:51.801958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1201/2382] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:08:51 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:08:51.809114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:51.818966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450fc97-c847-417f-a826-f5225f8d6287 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:51.890980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:51.900081 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:51.900701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:51.900984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:51.901258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:51.901890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:51.903830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:51.996277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1182/2383] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:08:51 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:08:51.999360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:52.009718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a934-6046-48ff-908b-31620b09217c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:52.016193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:52.016809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:52.017095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:52.017876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:52.018537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:52.020232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:52.031608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1202/2384] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:08:52.036528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:52.047281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb784e8-1783-4897-9287-f79cddbe2d63 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:52.052007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:52.052650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:52.052923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:52.053191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] PATH_INFO: `/v3/projects/346242baa82d43418f6f5f5307adf19b/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:52.068344 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] RBAC: Authorizing `identity:create_grant(project_id=346242baa82d43418f6f5f5307adf19b, user_id=ecfbd7fe60344a4b9f8a751b4ccbc1ef, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:52.070995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:52.152743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1183/2385] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:08:52 2022] PUT /identity/v3/projects/346242baa82d43418f6f5f5307adf19b/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:08:52.156738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:52.167341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef55c74-9979-4449-94d7-39b92bf1aed0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:52.213024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:08:52.227194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:52.227612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:52.227951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:52.228202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:52.228850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:52.230580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:52.240987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1203/2386] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:08:52.245351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:52.256923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e304385-8731-445b-8710-7eac0159925b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:52.261735 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:52.262408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:52.262786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:52.262943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] PATH_INFO: `/v3/projects/346242baa82d43418f6f5f5307adf19b/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:52.265780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346242baa82d43418f6f5f5307adf19b, user_id=ecfbd7fe60344a4b9f8a751b4ccbc1ef, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:52.269038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39024de4-3560-4a63-ab11-8c52779f6d53 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:52.285838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1184/2387] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:08:52 2022] PUT /identity/v3/projects/346242baa82d43418f6f5f5307adf19b/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:08:55.750602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:08:55.753485 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:55.922634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2387b3-a1d1-4179-b964-f668823852fe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:08:56.131658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:08:56.132822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:08:56.133318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:08:56.133779 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:08:56.136395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:08:56.229161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:08:56.231055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:08:56.258913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1204/2388] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:08:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:00.401590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1253eb-707c-409d-a903-34cbc26023f1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:00.412004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1253eb-707c-409d-a903-34cbc26023f1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:00.414663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1253eb-707c-409d-a903-34cbc26023f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:00.464911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ca1253eb-707c-409d-a903-34cbc26023f1 None None] MFA Rules not processed for user `ecfbd7fe60344a4b9f8a751b4ccbc1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:00.466515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca1253eb-707c-409d-a903-34cbc26023f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:00.552251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1185/2389] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:09:00.567102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:00.584222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2bea-4249-4bc1-a26b-bc8ec4c2485d service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:00.600997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:00.602426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:00.602917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:00.603429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:00.626508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:00.677839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:00.679389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:00.704593 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1205/2390] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:05.306416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:05.306850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:05.306946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:05.331966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] MFA Rules not processed for user `bb18a00d44ed4af186bdc589e1e14e22`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:05.333539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:05.403945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1186/2391] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:05.882016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:05.883503 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:05.946278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe3aa-c5ab-4bb5-9e50-6a86b09183a7 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:05.955538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:05.955538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:05.955538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:05.955538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:05.957744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:05.969153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:05.970991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:05.979988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1206/2392] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:06.388536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:06.390458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:06.448057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:06.482687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e2514-5383-447a-a6ef-0d75b7fe9886 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:06.488179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:06.488776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:06.492893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:06.492893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:06.492893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:06.503347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:06.505058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:06.515945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1187/2393] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:11.523729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:11.549448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9b4e7-23a7-45f4-9c38-48f13ecee14c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:11.557904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:11.571330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:11.574312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:11.574713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:11.577612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:11.629193 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:11.633269 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:11.655321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1207/2394] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:12.861544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:12.871238 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c1a9-02a7-4d5e-8816-77bfe0e1517b service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:12.875053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:12.875667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:12.876004 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:12.876276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:12.878452 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:12.889140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:12.891061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849855-5fd5-4c98-a3c8-19c94fdf896c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:12.899497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1188/2395] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:14.071006 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:14.073584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:14.093701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3707970-cda8-4c81-bf80-fad148af0a16 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:14.100584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:14.101199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:14.101487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:14.101736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:14.103706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:14.114154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a81d5-d145-40b4-83a9-ed28efff8f61 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:14.114660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a81d5-d145-40b4-83a9-ed28efff8f61 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:14.114759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a81d5-d145-40b4-83a9-ed28efff8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:14.118135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:14.123409 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:14.132436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1208/2396] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:14.135583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf4a81d5-d145-40b4-83a9-ed28efff8f61 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:14.137120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf4a81d5-d145-40b4-83a9-ed28efff8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:14.200866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1189/2397] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:09:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:09:14.370591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:14.370591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:14.444496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409947af-4a30-45e2-83b3-017aaa35acf4 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:14.449519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:14.450121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:14.450413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:14.450665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:14.459238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:14.460910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:14.469132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1209/2398] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:09:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:09:21.876616 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:21.877464 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:21.877893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:21.901956 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] MFA Rules not processed for user `2a9dbe3b7fdd482681b6ab94c26f4e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:21.903603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:21.912105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1190/2399] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:22.265887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:22.276038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0039148-1ee0-4c21-9c93-1d5eb7b9017b service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:22.280384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:22.282232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:22.282524 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:22.282837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:22.284820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:22.296025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:22.298200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:22.308191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1210/2400] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:22.529299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:22.553566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802eccd-6732-4591-b99f-f0b590fc500b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:22.560097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:22.560913 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:22.561292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:22.561567 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:22.565255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:22.688899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:22.690532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:22.727435 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1191/2401] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:23.837002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:23.838142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:23.856338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5860e70-9752-4515-b3e7-fa667720cb0a service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:23.870047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:23.871008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:23.871532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:23.872151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] PATH_INFO: `/v3/users/dfaf5a716c5b448f8e1274de2fad307c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:23.874836 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfaf5a716c5b448f8e1274de2fad307c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:23.877007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:23.883124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:24.119522 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfaf5a716c5b448f8e1274de2fad307c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.120230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfaf5a716c5b448f8e1274de2fad307c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.134215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfaf5a716c5b448f8e1274de2fad307c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.212109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfaf5a716c5b448f8e1274de2fad307c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.222373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfaf5a716c5b448f8e1274de2fad307c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.229552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1211/2402] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:09:23 2022] DELETE /identity/v3/users/dfaf5a716c5b448f8e1274de2fad307c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:24.539931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:24.552594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c68ed8-3e17-4ece-95c9-d6a7123eed93 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:24.612755 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:24.629073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:24.629706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:24.630114 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:24.630396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] PATH_INFO: `/v3/projects/233f1c55781b4099be6e250e664c4fb9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:24.631525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=233f1c55781b4099be6e250e664c4fb9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:24.633494 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:24.754846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233f1c55781b4099be6e250e664c4fb9', 'request_id': 'req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.762360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233f1c55781b4099be6e250e664c4fb9', 'request_id': 'req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.830143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233f1c55781b4099be6e250e664c4fb9', 'request_id': 'req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.845181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] The token cache is being invalidate because project 233f1c55781b4099be6e250e664c4fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:09:24.845623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.846322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:24.851251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1192/2403] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:09:24 2022] DELETE /identity/v3/projects/233f1c55781b4099be6e250e664c4fb9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:24.852492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:24.854218 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:24.874930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5c39c-d871-49f0-9b40-67454a8a952d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:24.957777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:24.958905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:24.959338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:24.959850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] PATH_INFO: `/v3/users/4b403696590747a9959a9321b259b1f3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:24.961332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b403696590747a9959a9321b259b1f3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:24.963941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:25.269928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b403696590747a9959a9321b259b1f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.271059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b403696590747a9959a9321b259b1f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.290063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b403696590747a9959a9321b259b1f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.349585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b403696590747a9959a9321b259b1f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.370279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70451dd6-12b9-4583-b3a5-5c66e33712ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b403696590747a9959a9321b259b1f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.380613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1212/2404] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:09:24 2022] DELETE /identity/v3/users/4b403696590747a9959a9321b259b1f3 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:25.648410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:25.650132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:25.664419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa7f6-97d5-419f-aae8-4b8ac1187b8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:25.747788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:25.748431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:25.748982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:25.749446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] PATH_INFO: `/v3/projects/530302527b314777b0ac3241328098e6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:25.752597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530302527b314777b0ac3241328098e6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:25.754622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:25.903178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530302527b314777b0ac3241328098e6', 'request_id': 'req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:25.909946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530302527b314777b0ac3241328098e6', 'request_id': 'req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:26.013280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530302527b314777b0ac3241328098e6', 'request_id': 'req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:26.025358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] The token cache is being invalidate because project 530302527b314777b0ac3241328098e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:09:26.025704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:26.026386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8ada328-ea88-4be0-a1ee-4a33bb028bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:26.029187 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1193/2405] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:09:25 2022] DELETE /identity/v3/projects/530302527b314777b0ac3241328098e6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:26.034663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.035247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.035441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.045205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:26.075588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:26.075588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:26.154940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1213/2406] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:26.156310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.156857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.158126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.168446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:26.186075 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:26.187594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:26.198858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1194/2407] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:09:26.203245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.214250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd3839-700d-451f-b570-20b149d90d81 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.220743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.221372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.221720 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.222181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.222838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:09:26.223143 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.225014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.238668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1214/2408] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:09:26.243351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.253410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f5b3a-b132-4ade-ab40-a0e2cb60bd86 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.258776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.259343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.259620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.259927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.260572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.262474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.349619 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1195/2409] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:09:26 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:09:26.352225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.365294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5c11a-e7ba-4469-a22e-b8184961a0c9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.459753 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.461047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.462081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.462523 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.464333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.466982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.596731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.603086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1215/2410] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:09:26 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:09:26.635937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19089-ddc7-4c1d-9ab2-60413a2d9351 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.644377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.645360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.645649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.646496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.647252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.649771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.678296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1196/2411] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:26.684960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.702560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba26939-a054-4b92-ba33-c3b4aea7ee5a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.708865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.709891 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.710175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.710452 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.711433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.713388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6ce3d-d158-4595-8b32-3cab6b1b0b2b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.727816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1216/2412] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:26.731858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:26.749585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4be0-d7f1-4d65-b98f-edb8a78baf43 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:26.771549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:26.776105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.776634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.780214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] PATH_INFO: `/v3/projects/cea40881648b4df393fc35c07d8f68dd/users/7b2f7b56f6d94908bf00b3f21954776c/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.808306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea40881648b4df393fc35c07d8f68dd, user_id=7b2f7b56f6d94908bf00b3f21954776c, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:26.812138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:26.926286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1197/2413] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:09:26 2022] PUT /identity/v3/projects/cea40881648b4df393fc35c07d8f68dd/users/7b2f7b56f6d94908bf00b3f21954776c/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:26.928310 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:26.928846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:26.929272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:26.952027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:26.954366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:27.047805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1217/2414] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:27.065908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:27.068555 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:27.151037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a97d81-fd89-40cd-8d3d-0967d349baef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:27.169115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:27.169115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:27.171215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:27.171215 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:27.187933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:27.190213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:27.215626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1198/2415] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:27.336472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:27.338427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:27.361616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3dd6c-56e3-4fdd-9f08-6dfb721dfa26 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:27.366976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:27.369888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:27.369888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:27.369888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] PATH_INFO: `/v3/users/684d59041e82466d8aad6d2964842bf5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:27.369888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=684d59041e82466d8aad6d2964842bf5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:27.372191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:27.376092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:27.595128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684d59041e82466d8aad6d2964842bf5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:27.595751 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684d59041e82466d8aad6d2964842bf5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:27.614442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684d59041e82466d8aad6d2964842bf5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:27.708893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684d59041e82466d8aad6d2964842bf5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:27.716832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4abfe2d7-5bbd-43ca-8dd7-50734f7a5ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684d59041e82466d8aad6d2964842bf5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:27.722949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1218/2416] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:09:27 2022] DELETE /identity/v3/users/684d59041e82466d8aad6d2964842bf5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:28.647055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:28.673588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33789f-98c9-47a2-b03b-c35a3e83fc28 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:28.775222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:28.775925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:28.776725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:28.776725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] PATH_INFO: `/v3/projects/e8751d4bd74240db9b57b7e9f788df94` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:28.777952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8751d4bd74240db9b57b7e9f788df94)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:28.780756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:28.945984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:28.954453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8751d4bd74240db9b57b7e9f788df94', 'request_id': 'req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:28.966291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8751d4bd74240db9b57b7e9f788df94', 'request_id': 'req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:29.058594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8751d4bd74240db9b57b7e9f788df94', 'request_id': 'req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:29.078159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] The token cache is being invalidate because project e8751d4bd74240db9b57b7e9f788df94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:09:29.078513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:29.080144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-82f87187-3e7a-4dbe-8807-faf5d12b1f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:09:29.084650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1199/2417] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:09:28 2022] DELETE /identity/v3/projects/e8751d4bd74240db9b57b7e9f788df94 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:29.089535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.090374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.090861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.121676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:29.123131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:29.203733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1219/2418] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:29.211536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.216025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.216025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.238563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:29.240516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:29.254632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1200/2419] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:29.256980 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.268643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6ec3e-8a8c-4f42-bd1c-ec34efcd9a6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.274492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.275215 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.275639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.276275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.277076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:09:29.277496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.281008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.304273 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1220/2420] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:09:29.304992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.315161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd1246-7246-44a8-9a6c-05100a7f1763 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.324853 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.326115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.326858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.326858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.328016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.332014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.482034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1201/2421] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:09:29 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:09:29.483110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.494787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073f73c-1091-4c91-89de-497e13539eb3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.560543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.561216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.561521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.561790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.562420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.564532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.676125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:29.687863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1221/2422] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:09:29 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:09:29.690643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.705152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eff64-f15b-4281-a2ef-e8038741d59f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.713726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.714989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.715273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.715572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.716272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.717931 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.728991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1202/2423] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:29.733370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.747736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00b022-f869-42ff-9de1-ca7de5f31d40 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.760516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.761165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.761436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.761809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.762471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.764133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8324d-3da9-488e-8bb4-de2018873c40 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.781182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1222/2424] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:29.782817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:29.797028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65547380-9957-4800-a3f3-a90032fee90b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:29.804987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:29.805639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.805918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.806195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] PATH_INFO: `/v3/projects/c56bd2a662e34a6395117027b5c28c5b/users/08d81bcd5a4f43ce8b4839ee15cb163b/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:29.826649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56bd2a662e34a6395117027b5c28c5b, user_id=08d81bcd5a4f43ce8b4839ee15cb163b, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:29.828481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:29.980547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1203/2425] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:09:29 2022] PUT /identity/v3/projects/c56bd2a662e34a6395117027b5c28c5b/users/08d81bcd5a4f43ce8b4839ee15cb163b/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:29.985775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc24c-047a-4c17-aefb-95993b335bbc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:29.986055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc24c-047a-4c17-aefb-95993b335bbc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:29.986330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc24c-047a-4c17-aefb-95993b335bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:30.008424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6d0dc24c-047a-4c17-aefb-95993b335bbc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:30.011705 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d0dc24c-047a-4c17-aefb-95993b335bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:30.098871 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1223/2426] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:30.111262 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:30.114608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:30.237937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7d099-17b8-4d19-83fa-f9315d0a489c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:30.254522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:30.255448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:30.257434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:30.258060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:30.272341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:30.275235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:30.291117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1204/2427] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:34.858387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:34.858880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:34.858978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:34.923222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] MFA Rules not processed for user `7b2f7b56f6d94908bf00b3f21954776c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:34.925087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:35.035208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1224/2428] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:35.035951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:35.037574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:35.064512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0a1d7-4c63-4343-80d1-1954b0bda06d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:35.098732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:35.099358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:35.108128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:35.108128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:35.108128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:35.108128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:35.197072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1205/2429] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:09:35 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:09:35.200964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:35.223499 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675b82-af4d-4b30-95b0-d61e2a5ef246 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:35.347968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:35.353477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:35.353477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:35.353477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:35.353477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:35.353477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:35.563884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1225/2430] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:09:35 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:09:35.568255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:35.580532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25748-6f2c-406d-8a9c-5724c2d4c11c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:35.591042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:35.592402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:35.592673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:35.593015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:35.593734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:35.595597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:35.611864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1206/2431] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:35.615943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:35.642305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659cb12-11a1-4064-86f3-5129f040cbe4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:35.657793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:35.658488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:35.658773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:35.659050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] PATH_INFO: `/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267/users/6fa2f8b2921f487b903d8ccff589950a/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:35.664104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:35.704942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa5a06a02f7496cb5e7fcdb4efc4267, user_id=6fa2f8b2921f487b903d8ccff589950a, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:35.708588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:35.810203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1226/2432] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:09:35 2022] PUT /identity/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267/users/6fa2f8b2921f487b903d8ccff589950a/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:35.811529 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:35.823125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99ee6a-9ac8-40d0-b99e-9b702f6db9c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:35.917713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:35.990379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:35.991149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:35.991432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:35.991707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:35.992411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:36.002727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:36.031942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1207/2433] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:09:36.032224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:36.059230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21140769-df85-4f4d-b5dd-16e0a7854f8b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:36.070573 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:36.071265 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:36.071547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:36.071851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] PATH_INFO: `/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267/users/6fa2f8b2921f487b903d8ccff589950a/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:36.079597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa5a06a02f7496cb5e7fcdb4efc4267, user_id=6fa2f8b2921f487b903d8ccff589950a, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:36.082729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f0467-4e27-4863-b870-3ec9bf1bb1bf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:36.122553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1227/2434] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:09:36 2022] PUT /identity/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267/users/6fa2f8b2921f487b903d8ccff589950a/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:09:36.619113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:36.621628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:36.757580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad8c0c-d4e6-4115-b55d-0dfd08a50b9f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:36.842291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:36.843322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:36.843704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:36.844203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:36.846111 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:36.940251 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:36.945560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:36.980566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1208/2435] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:37.656125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8029c8-a77c-44e4-ac33-3710965cda0e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:37.656604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8029c8-a77c-44e4-ac33-3710965cda0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:37.656700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8029c8-a77c-44e4-ac33-3710965cda0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:37.701666 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cc8029c8-a77c-44e4-ac33-3710965cda0e None None] MFA Rules not processed for user `08d81bcd5a4f43ce8b4839ee15cb163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:37.703278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc8029c8-a77c-44e4-ac33-3710965cda0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:37.863699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1228/2436] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:37.884994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:37.902745 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf894f-0203-4302-94d6-db730f20a7b4 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:37.909933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:37.910572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:37.911142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:37.911142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:37.921603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:37.926188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:37.928064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a190-273f-48d8-8f1e-9747f4ad1dba service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:37.938835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1209/2437] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:41.141448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:41.142345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:41.142799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:41.168299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] MFA Rules not processed for user `08d81bcd5a4f43ce8b4839ee15cb163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:41.171165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:41.181125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1229/2438] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:43.914491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:43.918129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:43.918129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:43.945090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] MFA Rules not processed for user `6fa2f8b2921f487b903d8ccff589950a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:43.946352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:44.087112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:44.107266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b63c1-389f-4293-86db-49621cc7f7fc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:44.116319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:44.120293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:44.124745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:44.124745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:44.126353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:44.127932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1210/2439] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:44.128985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:44.130517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:44.142275 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:44.144242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584582c9-b8da-49aa-879b-1f36b2eef2d3 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:44.148717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d82db9-883d-44fd-aa08-e759942cb3c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:44.157763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:44.157932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:44.158223 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:44.159056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:44.170299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1230/2440] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:09:44.230525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:44.233493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:44.262036 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1211/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:47.653197 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237390e-bfa9-4175-baf9-41fbb472aa1d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:47.653780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237390e-bfa9-4175-baf9-41fbb472aa1d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:47.653780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237390e-bfa9-4175-baf9-41fbb472aa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:47.687502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6237390e-bfa9-4175-baf9-41fbb472aa1d None None] MFA Rules not processed for user `7b2f7b56f6d94908bf00b3f21954776c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:47.688770 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6237390e-bfa9-4175-baf9-41fbb472aa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:47.696191 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1231/2442] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:09:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:48.049179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:48.108511 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d45b7-f81c-4649-9641-020dd9918802 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:48.112536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:48.113190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:48.113411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:48.113687 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:48.115442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:48.149835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:48.185395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:48.187255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:48.198398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1212/2443] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:09:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:09:57.668738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:58.224825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=14 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:09:59.617195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e48b-21ea-4cf4-9af7-fddfc0a7d6b7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:59.617657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e48b-21ea-4cf4-9af7-fddfc0a7d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:59.618088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1e48b-21ea-4cf4-9af7-fddfc0a7d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:59.641825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4c1e48b-21ea-4cf4-9af7-fddfc0a7d6b7 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:09:59.643045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4c1e48b-21ea-4cf4-9af7-fddfc0a7d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:59.712929 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1232/2444] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:09:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:09:59.736426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:09:59.736426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:09:59.935487 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028efa8-2b89-4681-b1a7-cbcd65001c2e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:09:59.941300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:09:59.941907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:09:59.942216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:09:59.942474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:09:59.957041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:09:59.958731 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2aefa7-962d-414c-8d5f-bad382735c60 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:09:59.971946 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1213/2445] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:09:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:10:09.700391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=25 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:09.965198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=32 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:18.861095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:18.861564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:18.861681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:18.862934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1233/2446] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 11:10:18.869524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc59e-64ab-40d4-a6c7-b93125b50fee None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:18.870658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc59e-64ab-40d4-a6c7-b93125b50fee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:18.871138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cc59e-64ab-40d4-a6c7-b93125b50fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:18.922560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c78cc59e-64ab-40d4-a6c7-b93125b50fee None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:18.923747 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c78cc59e-64ab-40d4-a6c7-b93125b50fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:18.971647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1214/2447] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:18.983093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:18.984754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:19.042076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff602bb-618b-43f2-a93d-8f527c5e1434 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:19.045610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:19.046212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.046695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.046977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.057305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:19.058680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e0c9b-2e3b-4c15-97a4-c9f354b0cbc9 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:19.071436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1234/2448] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:19.278089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.278515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.278858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.280305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1215/2449] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:10:19.285015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ad5b4-507a-45bd-93e0-b6219347e67d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.285406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ad5b4-507a-45bd-93e0-b6219347e67d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.285743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ad5b4-507a-45bd-93e0-b6219347e67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.321046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-129ad5b4-507a-45bd-93e0-b6219347e67d None None] MFA Rules not processed for user `c3ab7cc322d04f84ab56c54cf8429596`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:19.322783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-129ad5b4-507a-45bd-93e0-b6219347e67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:19.338295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1235/2450] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:19.350233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:19.351933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:19.366163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744267bd-0680-4e4d-bb59-3268872993ba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:19.370246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:19.370912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.371188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.371458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.378662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:19.380134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67cc57-e05f-4fef-bafd-b1e696cd3d86 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:19.388688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1216/2451] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:19.479214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace46d-debd-442f-9627-855036ad68e8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.479498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace46d-debd-442f-9627-855036ad68e8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.479790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace46d-debd-442f-9627-855036ad68e8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.481149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1236/2452] 10.222.0.48 () {54 vars in 921 bytes} [Thu Aug 11 11:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 11:10:19.486706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7910cf-3666-4d1b-ab10-5f1c222cc162 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.487517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7910cf-3666-4d1b-ab10-5f1c222cc162 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.487820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7910cf-3666-4d1b-ab10-5f1c222cc162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.510817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c7910cf-3666-4d1b-ab10-5f1c222cc162 None None] MFA Rules not processed for user `d622a3b3cc2a4e469033cb2d5c66d2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:19.512301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c7910cf-3666-4d1b-ab10-5f1c222cc162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:19.522523 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1217/2453] 10.222.0.48 () {58 vars in 1018 bytes} [Thu Aug 11 11:10:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:19.533461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace46d-debd-442f-9627-855036ad68e8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:19.544831 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace46d-debd-442f-9627-855036ad68e8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:19.548416 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:19.549015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:19.549295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:19.549563 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:19.560480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:19.564154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:19.573426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1237/2454] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:22.106339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:22.106831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:22.106942 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:22.126693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:22.127923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:22.135128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1218/2455] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:10:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:22.156912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:22.168774 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5835b8-2fcb-4791-bbaf-044dbf7cbde0 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:22.173672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:22.174262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:22.174537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:22.174805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:22.183327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:22.185110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5550f-3d33-4f89-a979-98be00984848 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:22.193133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1238/2456] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:10:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:10:32.121213 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=45 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:32.189850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=50 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:42.291280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:42.300539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062567dd-6484-4062-851f-23c5e10bf5a5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:42.304411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:42.305012 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:42.305289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:42.305559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:42.315345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:42.316803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:42.326139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1219/2457] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:43.038423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:43.038423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:43.038423 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:43.050675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:43.052130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:43.059734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1239/2458] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:10:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:10:43.073231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:43.085428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e8ee9-6915-4899-abb8-3c9667c87b52 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:43.091442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:43.092150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:43.094865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:43.094865 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:43.102032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:43.104089 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:43.112471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1220/2459] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:10:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:10:52.949953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:52.959252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9b094-c0e5-474d-a659-6a1c722c4982 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:52.963451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:52.964078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:52.964353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:52.964618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:52.966306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:53.047302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:53.048760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:53.070039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1240/2460] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:53.109323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=12 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:56.749316 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:56.751414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:56.848776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ef7f1-cf04-4983-9e4c-c10ff6445bea service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:56.854219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:56.854880 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:56.855167 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:56.855456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:56.857292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:56.870045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:56.871714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:56.883608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:56.886015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:56.886497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1221/2461] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:56.898496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1495d-74ce-4d21-8c06-a5d3dde77b71 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:56.903176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:56.903838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:56.904292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:56.904852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] PATH_INFO: `/v3/users/bb18a00d44ed4af186bdc589e1e14e22` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:56.906507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb18a00d44ed4af186bdc589e1e14e22)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:56.908640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:57.117372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb18a00d44ed4af186bdc589e1e14e22'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.117981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb18a00d44ed4af186bdc589e1e14e22'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.130249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb18a00d44ed4af186bdc589e1e14e22'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.198940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb18a00d44ed4af186bdc589e1e14e22'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.206478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c41d763f-cb9b-4ff5-a17e-1e4c85df0bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb18a00d44ed4af186bdc589e1e14e22'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.216014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1241/2462] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:10:56 2022] DELETE /identity/v3/users/bb18a00d44ed4af186bdc589e1e14e22 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:10:57.497958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:57.512054 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ee91e-bd34-43c6-9490-0d326dba4c84 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:57.574600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:57.575312 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:57.575594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:57.575944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] PATH_INFO: `/v3/projects/b0b3799012d24fad98d155e3b000fc22` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:57.577071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b3799012d24fad98d155e3b000fc22)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:57.579348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:57.692176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0a7bf-d3b3-4769-a2f0-6cced5cbcacc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:57.692465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0a7bf-d3b3-4769-a2f0-6cced5cbcacc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:57.696004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0a7bf-d3b3-4769-a2f0-6cced5cbcacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:57.697577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b3799012d24fad98d155e3b000fc22', 'request_id': 'req-316c23e1-fcc6-4195-af0c-a86e21f61ab1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.704944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b3799012d24fad98d155e3b000fc22', 'request_id': 'req-316c23e1-fcc6-4195-af0c-a86e21f61ab1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.712406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-24f0a7bf-d3b3-4769-a2f0-6cced5cbcacc None None] MFA Rules not processed for user `08d81bcd5a4f43ce8b4839ee15cb163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:57.713597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-24f0a7bf-d3b3-4769-a2f0-6cced5cbcacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:57.739122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:57.766608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1242/2463] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:57.822649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b3799012d24fad98d155e3b000fc22', 'request_id': 'req-316c23e1-fcc6-4195-af0c-a86e21f61ab1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.839513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] The token cache is being invalidate because project b0b3799012d24fad98d155e3b000fc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:10:57.839923 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.840672 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-316c23e1-fcc6-4195-af0c-a86e21f61ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:10:57.842218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1222/2464] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:10:57 2022] DELETE /identity/v3/projects/b0b3799012d24fad98d155e3b000fc22 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:10:57.847130 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:57.847430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:57.847692 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:57.895612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:57.896884 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:57.978165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1243/2465] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:57.986964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:57.987250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:57.987527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.007037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:58.008395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:58.016445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1223/2466] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:58.020468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.030706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c44f-62cf-44ff-a70a-5020271fdc2c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.035334 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.036145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.036283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.036596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.037238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:10:58.037542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.039443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.053281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1244/2467] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:10:58.057845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.068095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c41e2-c4cf-4c18-b054-6fc6143d4ca7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.073372 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.073975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.074253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.074764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.075395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.077336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.091206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:58.180172 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1224/2468] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:10:58 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:10:58.185005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.196981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e9622-2082-4815-b609-8dcfa142205b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.259710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.260805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.261095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.261373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.262001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.264542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.371163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1245/2469] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:10:58 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:10:58.375345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.386030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdb111-6c27-4401-8192-9c72a3e650dc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.390565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.391162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.391441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.391719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.392771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.399640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.404684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1225/2470] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:10:58.408737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.419360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286b24-6515-487b-abbb-4bf18e7745da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.424023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.425955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.425955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.425955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.425955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.427346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adad6cc-3e1f-40d4-a11a-61604569b7b9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.439707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1246/2471] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:10:58.443925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.454438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd1ea5-eb30-4808-884b-f2132499182a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.458926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.459554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.459860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.460131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] PATH_INFO: `/v3/projects/7cec53327f8e4f499abbce42a1097764/users/cefd24fa1911424db65cf48bc2465610/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.474577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec53327f8e4f499abbce42a1097764, user_id=cefd24fa1911424db65cf48bc2465610, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.477214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.561940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1226/2472] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:10:58 2022] PUT /identity/v3/projects/7cec53327f8e4f499abbce42a1097764/users/cefd24fa1911424db65cf48bc2465610/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:10:58.566793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b825-bc40-4651-8a29-ea1a63fba513 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.569594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b825-bc40-4651-8a29-ea1a63fba513 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.569594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b825-bc40-4651-8a29-ea1a63fba513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.598864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f522b825-bc40-4651-8a29-ea1a63fba513 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:58.600097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f522b825-bc40-4651-8a29-ea1a63fba513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:58.614654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:10:58.662706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1247/2473] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:58.673312 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:58.675122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:58.737006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd8ae7-d92c-4681-9fd0-0a4bfd32aa6b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:58.741136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:58.742514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:58.742799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:58.743073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:58.753508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:58.754124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:10:58.763989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1227/2474] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:10:59.864538 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:59.865047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:59.865308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:59.894116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] MFA Rules not processed for user `cefd24fa1911424db65cf48bc2465610`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:10:59.895387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:10:59.969426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1248/2475] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:10:59.973920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:10:59.984556 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a252-060e-4a2b-bbb3-9bc9df289c81 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:10:59.994377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:10:59.994981 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:10:59.995255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:10:59.995522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:10:59.996220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:10:59.998323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.075138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1228/2476] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:10:59 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:00.079326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:00.090832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d7ef7-b2de-4268-901c-e8471c1bb960 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:00.147021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:00.147541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:00.148483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:00.148760 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:00.149412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:00.151322 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.250713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1249/2477] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:00 2022] POST /identity/v3/users => generated 309 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:00.255659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:00.267178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7a36b-220a-4107-b767-018affbd621e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:00.271709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:00.272345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:00.273250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:00.273536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:00.274207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:00.275796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.286307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1229/2478] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:00.290971 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:00.302326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee140006-b60b-4bc8-9571-15f47e311258 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:00.306925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:00.307564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:00.307865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:00.308139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] PATH_INFO: `/v3/projects/2a13287ca06541fa94643de804a93ab4/users/98032108700f46d09ace73753e3f4892/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:00.323404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a13287ca06541fa94643de804a93ab4, user_id=98032108700f46d09ace73753e3f4892, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:00.326286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.388548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1250/2479] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:00 2022] PUT /identity/v3/projects/2a13287ca06541fa94643de804a93ab4/users/98032108700f46d09ace73753e3f4892/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:11:00.392906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:00.403625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6708440-8b65-4b48-b6f8-078c3439e0da admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:00.459477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:00.460116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:00.461145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:00.461430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:00.462072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:00.463693 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb31f8-49a9-4f8d-bf39-106b9ca8e61d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.473927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1230/2480] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:00.478254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:00.488317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9daec7-71a3-41bf-885e-7295551ee311 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:00.494003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:00.494640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:00.494997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:00.495271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] PATH_INFO: `/v3/projects/2a13287ca06541fa94643de804a93ab4/users/98032108700f46d09ace73753e3f4892/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:00.497185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a13287ca06541fa94643de804a93ab4, user_id=98032108700f46d09ace73753e3f4892, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:00.499801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:00.512134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1251/2481] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:00 2022] PUT /identity/v3/projects/2a13287ca06541fa94643de804a93ab4/users/98032108700f46d09ace73753e3f4892/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:02.155148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:02.155724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:02.155724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:02.183704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:02.187404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] MFA Rules not processed for user `98032108700f46d09ace73753e3f4892`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:02.188761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:02.264812 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1231/2482] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:03.824187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:03.826013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:03.831327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:03.897118 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7204a4-50f8-4303-bd57-50175da8c23c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:03.903226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:03.903863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:03.904141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:03.904413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:03.906103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:03.917569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:03.919225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:03.929200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1252/2483] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:03.996058 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:04.059708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766597c-ca51-40a7-ad56-041dfdef6e91 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:04.064269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:04.064888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:04.065169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:04.065446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] PATH_INFO: `/v3/users/cefd24fa1911424db65cf48bc2465610` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:04.066575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cefd24fa1911424db65cf48bc2465610)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:04.068571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:04.234354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cefd24fa1911424db65cf48bc2465610'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:04.235009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cefd24fa1911424db65cf48bc2465610'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:04.242226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cefd24fa1911424db65cf48bc2465610'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:04.253784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cefd24fa1911424db65cf48bc2465610'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:04.306977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cefd24fa1911424db65cf48bc2465610'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:04.313178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1232/2484] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:11:03 2022] DELETE /identity/v3/users/cefd24fa1911424db65cf48bc2465610 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:04.863741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:04.878674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1961a74-2896-49f4-93c4-4db94594a8a2 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:04.934202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:04.934822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:04.935160 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:04.935429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] PATH_INFO: `/v3/projects/7cec53327f8e4f499abbce42a1097764` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:04.936728 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec53327f8e4f499abbce42a1097764)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:04.938613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:05.067111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec53327f8e4f499abbce42a1097764', 'request_id': 'req-5dde9484-7ee7-4784-9e96-faf018def408'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.073142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec53327f8e4f499abbce42a1097764', 'request_id': 'req-5dde9484-7ee7-4784-9e96-faf018def408'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.215777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec53327f8e4f499abbce42a1097764', 'request_id': 'req-5dde9484-7ee7-4784-9e96-faf018def408'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.226985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] The token cache is being invalidate because project 7cec53327f8e4f499abbce42a1097764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:11:05.227272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.227911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.229403 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1253/2485] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:11:04 2022] DELETE /identity/v3/projects/7cec53327f8e4f499abbce42a1097764 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:05.233400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:05.234785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:05.247723 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c398-6d9b-4847-8d0e-b44596a49ec8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:05.259856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:05.303930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:05.304550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:05.304830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:05.305103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] PATH_INFO: `/v3/users/98032108700f46d09ace73753e3f4892` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:05.306218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98032108700f46d09ace73753e3f4892)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:05.308162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:05.497158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98032108700f46d09ace73753e3f4892'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.497748 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98032108700f46d09ace73753e3f4892'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.504671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98032108700f46d09ace73753e3f4892'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.516847 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98032108700f46d09ace73753e3f4892'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.581722 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b161688f-b194-4ba5-aac7-1541a180b230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98032108700f46d09ace73753e3f4892'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:05.588569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1233/2486] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:11:05 2022] DELETE /identity/v3/users/98032108700f46d09ace73753e3f4892 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:05.868699 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:05.870366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:05.891395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde9484-7ee7-4784-9e96-faf018def408 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:05.914508 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:05.957591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:05.958231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:05.958505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:05.958776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] PATH_INFO: `/v3/projects/2a13287ca06541fa94643de804a93ab4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:05.959945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a13287ca06541fa94643de804a93ab4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:05.962066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.114206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a13287ca06541fa94643de804a93ab4', 'request_id': 'req-cdc27887-039e-49b6-962a-475148042513'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:06.120800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a13287ca06541fa94643de804a93ab4', 'request_id': 'req-cdc27887-039e-49b6-962a-475148042513'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:06.267366 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a13287ca06541fa94643de804a93ab4', 'request_id': 'req-cdc27887-039e-49b6-962a-475148042513'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:06.279231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] The token cache is being invalidate because project 2a13287ca06541fa94643de804a93ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:11:06.279590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:06.280318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc27887-039e-49b6-962a-475148042513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:06.281809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1254/2487] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:11:05 2022] DELETE /identity/v3/projects/2a13287ca06541fa94643de804a93ab4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:06.286825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.287309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.287418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.308632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:06.310357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:06.378142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1234/2488] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:11:06.387074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.387364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.387641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.407135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:06.408461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:06.417146 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1255/2489] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:06.421269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.431457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800a05b-eaae-4940-94ef-56cdf031e80a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.435964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.436548 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.436823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.437091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.437736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:11:06.438021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.439624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.450736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1235/2490] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:11:06.454851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.464471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8c6e0-6c7b-4d61-8e9b-c4916c873cec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.469200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.469802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.470181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.470369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.470974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.472907 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.670457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1256/2491] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:11:06 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:06.674252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.684188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173952ec-5b44-49d0-a8c8-9b0c244ee616 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.705446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:06.742806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.743411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.743695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.744009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.744631 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.746521 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.872920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1236/2492] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:06 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:06.877307 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.888565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66151aa8-20da-467d-b50d-d983b3056d3c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.893114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.893640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.893914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.894323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.894845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.896613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.907289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1257/2493] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:06.911357 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.921562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1f3a-867d-4802-b896-dd8a47e1791c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.926082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.929091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.929091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.929091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.929091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.929425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce49c3c-d0c9-4e0b-b464-2bd69e9a58f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:06.939179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1237/2494] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:06.943451 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:06.954092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc894-39b8-4815-bf78-b00ce64e3b82 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:06.958734 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:06.959371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:06.960293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:06.960564 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] PATH_INFO: `/v3/projects/a4b3543e331245afa5d9ca1520922082/users/fabff898319a45779c7a7d668e5cfd11/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:06.975238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b3543e331245afa5d9ca1520922082, user_id=fabff898319a45779c7a7d668e5cfd11, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:06.978025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.039428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1258/2495] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:06 2022] PUT /identity/v3/projects/a4b3543e331245afa5d9ca1520922082/users/fabff898319a45779c7a7d668e5cfd11/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:07.050088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.050379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.050657 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.079038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] MFA Rules not processed for user `fabff898319a45779c7a7d668e5cfd11`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:07.080308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:07.145074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1238/2496] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:07.150089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.200863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:07.213928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410963b4-1244-43aa-8745-83a0ceae60d2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.218518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.219112 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.219385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.219652 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.220294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.222206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.308610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1259/2497] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:11:07 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:07.312738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.323850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4883d-a23a-4cd6-a945-3d70bf42b619 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.386527 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.387163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.387440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.387710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.388395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.390292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.489928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1239/2498] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:07 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:07.494542 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.504847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c128c26-8066-42b8-98d3-537d7bdb5c8e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.510075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.510662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.510941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.511212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.511926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.513509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.523242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1260/2499] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:07.527260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.537033 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5b0b4-c78f-4aeb-9373-0d6ff7d86e0b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.542120 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.542760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.543042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.543319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] PATH_INFO: `/v3/projects/335fceab8fc44aa7b58bc0546d263306/users/761c1c81854545e1b2cbe40f32c4d725/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.558481 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=335fceab8fc44aa7b58bc0546d263306, user_id=761c1c81854545e1b2cbe40f32c4d725, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.561153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.697059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:07.702575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1240/2500] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:07 2022] PUT /identity/v3/projects/335fceab8fc44aa7b58bc0546d263306/users/761c1c81854545e1b2cbe40f32c4d725/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:07.706690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.716592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee07d-f5dd-465e-b6bd-70e381c7c2d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.771793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.772351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.772627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.772896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.773529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.775141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70070-e4a8-4df7-9010-86bf73726167 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.784976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1261/2501] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:07.789826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.800108 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c3289-b3fe-45ee-8fdc-c093822429cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:07.805289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:07.805924 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.806204 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.806504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] PATH_INFO: `/v3/projects/335fceab8fc44aa7b58bc0546d263306/users/761c1c81854545e1b2cbe40f32c4d725/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.808446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335fceab8fc44aa7b58bc0546d263306, user_id=761c1c81854545e1b2cbe40f32c4d725, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:07.811059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:07.821059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1241/2502] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:07 2022] PUT /identity/v3/projects/335fceab8fc44aa7b58bc0546d263306/users/761c1c81854545e1b2cbe40f32c4d725/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:11:07.831255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:07.831545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:07.831859 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:07.859196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] MFA Rules not processed for user `761c1c81854545e1b2cbe40f32c4d725`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:07.860438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:07.928834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1262/2503] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:07.940326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:07.942008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:08.019395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc2d2-421d-4a6b-836b-dc3c3c8b4322 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:08.023602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:08.024168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:08.024445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:08.024714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:08.033578 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:08.034959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:08.045186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1242/2504] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:08.296339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:08.306479 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa2f27-f212-492a-878f-89fac9e69938 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:08.311702 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:08.312121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:08.312498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:08.313192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:08.364195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:08.380029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:08.380029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:08.389064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1263/2505] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:18.041103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:18.598507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:33.691900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:33.693881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:33.748881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5814a-da25-461e-a67a-79a91a85d93c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:33.752420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:33.753034 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:33.753310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:33.753582 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:33.755743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:33.815701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:33.817190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:33.827207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1243/2506] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:37.341207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:37.342650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:37.357302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58167d40-cd9a-4a46-98f9-696ec45d2f8d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:37.361858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:37.362483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:37.362762 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:37.363035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] PATH_INFO: `/v3/users/1b7724f3ef4543ed9c3df0455ae4fb76` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:37.364306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7724f3ef4543ed9c3df0455ae4fb76)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:37.366722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:37.530470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7724f3ef4543ed9c3df0455ae4fb76'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:37.530765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7724f3ef4543ed9c3df0455ae4fb76'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:37.543819 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7724f3ef4543ed9c3df0455ae4fb76'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:37.607909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7724f3ef4543ed9c3df0455ae4fb76'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:37.615999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1f08107b-9352-43e2-b60b-e1b49d1dc345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7724f3ef4543ed9c3df0455ae4fb76'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:37.622199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1264/2507] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:11:37 2022] DELETE /identity/v3/users/1b7724f3ef4543ed9c3df0455ae4fb76 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:37.850088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:37.860470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a804dc1-48ba-4547-b0b5-554768d9756d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:37.919179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:37.919822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:37.920105 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:37.920380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] PATH_INFO: `/v3/projects/c42a3b4ef2d644b58856779226688f88` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:37.921478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42a3b4ef2d644b58856779226688f88)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:37.923395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.053966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42a3b4ef2d644b58856779226688f88', 'request_id': 'req-b64f6ef5-4e51-41ba-855b-9215512f2db7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:38.055843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:38.060212 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42a3b4ef2d644b58856779226688f88', 'request_id': 'req-b64f6ef5-4e51-41ba-855b-9215512f2db7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:38.150604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42a3b4ef2d644b58856779226688f88', 'request_id': 'req-b64f6ef5-4e51-41ba-855b-9215512f2db7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:38.161405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] The token cache is being invalidate because project c42a3b4ef2d644b58856779226688f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:11:38.161691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:38.162558 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b64f6ef5-4e51-41ba-855b-9215512f2db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:38.164067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1244/2508] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:11:37 2022] DELETE /identity/v3/projects/c42a3b4ef2d644b58856779226688f88 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:38.169076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.169364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.169635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.190754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:38.195639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:38.259117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1265/2509] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:38.267110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.267392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.267688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.287674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:38.288891 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:38.296668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1245/2510] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:38.300724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.310233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c070cc9-32fa-462a-8d2a-aa25d7e83790 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.315007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.315630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.315945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.316219 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.316879 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:11:38.317684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.319342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.330714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1266/2511] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:11:38.335512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.344831 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c402-db70-4b17-8f76-0cd71ce14055 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.388692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.388692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.388692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.388692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.389123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.389123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.540577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1246/2512] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:11:38 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:38.544750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.555443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c434b3-3787-4e83-8676-e74136419e0d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.612179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.612788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.613068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.613342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.613958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.616396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.624089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:38.704417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1267/2513] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:38 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:38.708445 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.718695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c66a0-7fe5-4e79-a425-ab56eea1d59a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.723241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.723860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.724142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.724414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.725045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.726626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.736476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1247/2514] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:38.740764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.750387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a843d0-792b-4b41-ae10-10cbdd4a46ed admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.755440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.756054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.756331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.756601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.757226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.758802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6df868-0537-41b6-9338-08207a797091 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.771838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1268/2515] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:38.773225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:38.783124 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1817-e7d1-489b-9b19-95088e8554e4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:38.788233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:38.788917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.789194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.789495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] PATH_INFO: `/v3/projects/bc96075b63ca41be8f5be1fdea69b1fa/users/f69c530af741495bb30d0971d6e3adcf/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.804892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc96075b63ca41be8f5be1fdea69b1fa, user_id=f69c530af741495bb30d0971d6e3adcf, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:38.807610 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:38.910135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1248/2516] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:38 2022] PUT /identity/v3/projects/bc96075b63ca41be8f5be1fdea69b1fa/users/f69c530af741495bb30d0971d6e3adcf/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:38.921820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:38.929171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:38.929171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:38.950725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] MFA Rules not processed for user `f69c530af741495bb30d0971d6e3adcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:38.952078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:39.014803 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1269/2517] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:39.026067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:39.027780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:39.100383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7ae26-bde8-4d1c-8bed-374d46e9e3f2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:39.105893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:39.106489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:39.106770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:39.107045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:39.115302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:39.116898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f014c4d-9be7-45da-a7be-1785c1f035df service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:39.125514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1249/2518] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:45.390932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:45.454217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88642f-5423-414d-a6ed-b95ba3684e0b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:45.458912 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:45.459535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:45.460015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:45.460295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] PATH_INFO: `/v3/users/f69c530af741495bb30d0971d6e3adcf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:45.461418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69c530af741495bb30d0971d6e3adcf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:45.463380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:45.654817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69c530af741495bb30d0971d6e3adcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:45.655463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69c530af741495bb30d0971d6e3adcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:45.668263 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69c530af741495bb30d0971d6e3adcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:45.671987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:45.743928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69c530af741495bb30d0971d6e3adcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:45.750845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69c530af741495bb30d0971d6e3adcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:45.757084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1270/2519] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:11:45 2022] DELETE /identity/v3/users/f69c530af741495bb30d0971d6e3adcf => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:45.761658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:45.762117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:45.762222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:45.781052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:45.782241 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:45.839917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1250/2520] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:45.849987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:45.851356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:45.906408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8127a3f-6d7c-4bb7-a23e-9281cf49f2f4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:45.909930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:45.910519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:45.910791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:45.911059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:45.920536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:45.921934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa396d2-6a06-4179-8701-0390608a6c80 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:45.931722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1271/2521] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:46.334480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:46.336022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:46.343277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:46.349544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac9e5-fa93-4962-b4ca-e2273b971bb2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:46.354076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:46.354706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.354966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.355235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] PATH_INFO: `/v3/projects/bc96075b63ca41be8f5be1fdea69b1fa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.356411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc96075b63ca41be8f5be1fdea69b1fa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:46.358311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:46.482348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc96075b63ca41be8f5be1fdea69b1fa', 'request_id': 'req-b62901b9-53b3-4acc-a9a7-0987881b9884'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:46.488633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc96075b63ca41be8f5be1fdea69b1fa', 'request_id': 'req-b62901b9-53b3-4acc-a9a7-0987881b9884'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:46.584893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc96075b63ca41be8f5be1fdea69b1fa', 'request_id': 'req-b62901b9-53b3-4acc-a9a7-0987881b9884'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:46.597589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] The token cache is being invalidate because project bc96075b63ca41be8f5be1fdea69b1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:11:46.597879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:46.598571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b62901b9-53b3-4acc-a9a7-0987881b9884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:11:46.600152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1251/2522] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:11:46 2022] DELETE /identity/v3/projects/bc96075b63ca41be8f5be1fdea69b1fa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:46.605203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.605744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.605744 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.625103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:46.626296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:46.685469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1272/2523] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:46.693986 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.694274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.694546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.724712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:46.724712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:46.724712 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1252/2524] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:46.728079 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:46.737829 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933fc37-9847-4e55-ad47-a87a5e8d0b75 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:46.742963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:46.743552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.743856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.744144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.744763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:11:46.745058 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:46.746645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:46.758672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1273/2525] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:11:46.762978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:46.773820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0782564-4dea-4692-ba3f-34b6beaed5cc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:46.779671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:46.780300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.780580 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.780959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.781479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:46.783471 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:46.894593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1253/2526] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:11:46 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:46.898765 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:46.908721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6c573-e39c-40f6-8fe2-6b4e74319280 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:46.972528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:46.973596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:46.974059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:46.974445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:46.977928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:46.977928 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.080877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1274/2527] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:46 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:47.085061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.095384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20832c02-2dba-4f7c-890c-a9b97a5993f1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.100876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.101456 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.101732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.102000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.102630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.104269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.114461 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1254/2528] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:47.118969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.128851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186ec7-836e-42d7-bcb9-b71198f36e57 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.133498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.134093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.134374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.134647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.135590 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.137005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4d2fe-8b25-4e58-97b6-4d61f560e6e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.138247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:47.147984 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1275/2529] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:47.152263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.162757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e314b-47b0-4cd0-b23a-11396fa77b87 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.167475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.168151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.168432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.168710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] PATH_INFO: `/v3/projects/b1041c4f4f7b447db3056a62b9fdd2ba/users/7ed2fc2978c546b582b51dc0e4aa3abe/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.183783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1041c4f4f7b447db3056a62b9fdd2ba, user_id=7ed2fc2978c546b582b51dc0e4aa3abe, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.186467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.275508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1255/2530] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:47 2022] PUT /identity/v3/projects/b1041c4f4f7b447db3056a62b9fdd2ba/users/7ed2fc2978c546b582b51dc0e4aa3abe/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:47.286099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.286379 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.287450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.316337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] MFA Rules not processed for user `7ed2fc2978c546b582b51dc0e4aa3abe`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:47.317517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:47.380677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1276/2531] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:47.385299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.446362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b1616-4407-4c6c-a44e-ddb5473ef1f8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.450994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.451587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.451895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.452170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.452792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.454675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.551374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1256/2532] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:11:47 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:11:47.555292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.565978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be4281-6fe7-43e5-84c6-2cf3338b8797 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.624353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.624960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.625224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.628038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.628038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.628234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.710733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1277/2533] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:11:47 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:11:47.714882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.725739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3ba4e-c817-443f-a053-acf4de82f619 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.730365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.730951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.731255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.731501 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.732163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.733756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:47.734985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:47.745753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1257/2534] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:47.749885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:47.759707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aca4cf-22f4-4398-909f-33cea10fa5a4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:47.764346 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:47.764976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:47.765254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:47.765528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] PATH_INFO: `/v3/projects/47358a8232234c108bf4f74e6d72b6d0/users/4d679267f8914ae39cad0ca92467ce58/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:47.780365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47358a8232234c108bf4f74e6d72b6d0, user_id=4d679267f8914ae39cad0ca92467ce58, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:47.782991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:48.127550 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1278/2535] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:47 2022] PUT /identity/v3/projects/47358a8232234c108bf4f74e6d72b6d0/users/4d679267f8914ae39cad0ca92467ce58/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:48.131955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:48.141698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb2c5-6e94-47a0-a5de-d74cc3ff3277 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:48.195671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:48.196347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:48.196560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:48.196828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:48.197450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:48.199010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5bec5-224d-4fe5-b924-88061afad002 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:48.209312 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1258/2536] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:11:48.213407 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:48.223724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10ef51-e69e-43ed-a44e-1497485f8396 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:48.228514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:48.229159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:48.229433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:48.229703 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] PATH_INFO: `/v3/projects/47358a8232234c108bf4f74e6d72b6d0/users/4d679267f8914ae39cad0ca92467ce58/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:48.231548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47358a8232234c108bf4f74e6d72b6d0, user_id=4d679267f8914ae39cad0ca92467ce58, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:48.234267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:48.244886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1279/2537] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:11:48 2022] PUT /identity/v3/projects/47358a8232234c108bf4f74e6d72b6d0/users/4d679267f8914ae39cad0ca92467ce58/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:11:48.256675 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:48.261129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:48.261129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:48.283566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] MFA Rules not processed for user `4d679267f8914ae39cad0ca92467ce58`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:11:48.284784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:48.351369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1259/2538] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:11:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:11:48.362710 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:48.364442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:11:48.408618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:48.423525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d9210-f051-45ee-a4f5-1ab4f7eaa9e0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:48.427635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:48.428267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:48.428581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:48.428851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:48.437529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:48.438940 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:48.448885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1280/2539] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:48.856966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:11:48.866993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00937-347a-4f9c-8d5a-a0eac0aa7266 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:11:48.870581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:11:48.871171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:11:48.871475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:11:48.871725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:11:48.928842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:11:48.930332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c12bd-f2ef-4fb2-be9e-c02482abbda4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:11:48.939686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1260/2540] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:11:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:11:58.903997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=9 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:11:58.935271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:01.762333 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:01.825408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780aa24-c9b3-4aa5-b0b3-69b6171b813e service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:01.830041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:01.830651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:01.830926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:01.831192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] PATH_INFO: `/v3/users/7ed2fc2978c546b582b51dc0e4aa3abe` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:01.832332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed2fc2978c546b582b51dc0e4aa3abe)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:01.835103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:02.083974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed2fc2978c546b582b51dc0e4aa3abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:02.085238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed2fc2978c546b582b51dc0e4aa3abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:02.098397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed2fc2978c546b582b51dc0e4aa3abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:02.196076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed2fc2978c546b582b51dc0e4aa3abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:02.216723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed2fc2978c546b582b51dc0e4aa3abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:02.226743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1281/2541] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:01 2022] DELETE /identity/v3/users/7ed2fc2978c546b582b51dc0e4aa3abe => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:02.231130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:02.231934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:02.231934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:02.261181 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:02.262576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:02.335465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1261/2542] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:02.352001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:02.354873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:02.430000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481eeec5-25cf-4879-840b-32ef6352a242 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:02.439874 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:02.440681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:02.441024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:02.442354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:02.457385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:02.459176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:02.475428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1282/2543] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:03.269726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:03.272292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:03.290228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e025212-c063-49ec-aa3e-5c89cf6d5964 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:03.296135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:03.296787 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:03.297072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:03.297405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] PATH_INFO: `/v3/users/fabff898319a45779c7a7d668e5cfd11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:03.298897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fabff898319a45779c7a7d668e5cfd11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:03.302010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:03.543819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fabff898319a45779c7a7d668e5cfd11'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.544721 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fabff898319a45779c7a7d668e5cfd11'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.550278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:03.552882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:03.561673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fabff898319a45779c7a7d668e5cfd11'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.590077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fabff898319a45779c7a7d668e5cfd11'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.605124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e09d0-9651-4f79-8e45-7cbefe6c7e04 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:03.611645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:03.613386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:03.613386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:03.613386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] PATH_INFO: `/v3/projects/b1041c4f4f7b447db3056a62b9fdd2ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:03.614799 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1041c4f4f7b447db3056a62b9fdd2ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:03.617514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:03.649237 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:03.692268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0e94f365-18da-4fe5-9c7f-2b9221145768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fabff898319a45779c7a7d668e5cfd11'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.703810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1262/2544] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:03 2022] DELETE /identity/v3/users/fabff898319a45779c7a7d668e5cfd11 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:03.709153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:03.710587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:03.716186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:03.784991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:03.788485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:03.796417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1041c4f4f7b447db3056a62b9fdd2ba', 'request_id': 'req-73eada4a-922d-4747-8a35-33fc1f5bbcdc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.821022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1041c4f4f7b447db3056a62b9fdd2ba', 'request_id': 'req-73eada4a-922d-4747-8a35-33fc1f5bbcdc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.925681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1041c4f4f7b447db3056a62b9fdd2ba', 'request_id': 'req-73eada4a-922d-4747-8a35-33fc1f5bbcdc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.938423 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1263/2545] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:03.946271 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:03.947926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:03.948179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] The token cache is being invalidate because project b1041c4f4f7b447db3056a62b9fdd2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:03.948688 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.951909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:03.962380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1283/2546] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:03 2022] DELETE /identity/v3/projects/b1041c4f4f7b447db3056a62b9fdd2ba => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:03.966947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:03.977141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.009156 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:04.014347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eada4a-922d-4747-8a35-33fc1f5bbcdc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:04.031840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:04.032455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:04.032752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:04.033013 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] PATH_INFO: `/v3/users/4d679267f8914ae39cad0ca92467ce58` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:04.034179 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d679267f8914ae39cad0ca92467ce58)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:04.036454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:04.100446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476dd0f-9266-4137-bb6d-0b0591cdc4ca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:04.101132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:04.101808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:04.102098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:04.102365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:04.104321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.128627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:04.131974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:04.147832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1264/2547] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:04.378174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:04.379690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.384461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d679267f8914ae39cad0ca92467ce58'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.385947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d679267f8914ae39cad0ca92467ce58'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.442587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d679267f8914ae39cad0ca92467ce58'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.515629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d679267f8914ae39cad0ca92467ce58'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.526121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d679267f8914ae39cad0ca92467ce58'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.528933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043c343-df61-4398-ad4a-6be86c7dc3a1 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:04.541339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1284/2548] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:03 2022] DELETE /identity/v3/users/4d679267f8914ae39cad0ca92467ce58 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:04.549500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:04.550138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:04.550419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:04.550694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:04.553515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.568187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:04.569738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.577774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:04.579955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:04.601285 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1265/2549] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:04.675235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0e20f-9782-4037-a865-4953839c4df7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:04.677445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:04.678585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:04.679276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:04.679727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:04.682726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:04.684663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:04.706165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9d580-80af-456b-a3e8-ab529c169523 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:04.712125 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:04.716750 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:04.738060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1285/2550] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:04.812190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:04.812864 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:04.813145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:04.813419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] PATH_INFO: `/v3/projects/a4b3543e331245afa5d9ca1520922082` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:04.815352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b3543e331245afa5d9ca1520922082)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:04.817962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:04.946039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:04.964207 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b3543e331245afa5d9ca1520922082', 'request_id': 'req-e1b03d24-b613-46ea-bcf8-1e46c47c825c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:04.977176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b3543e331245afa5d9ca1520922082', 'request_id': 'req-e1b03d24-b613-46ea-bcf8-1e46c47c825c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.063410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b3543e331245afa5d9ca1520922082', 'request_id': 'req-e1b03d24-b613-46ea-bcf8-1e46c47c825c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.082972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] The token cache is being invalidate because project a4b3543e331245afa5d9ca1520922082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:05.082972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.084960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.086922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1266/2551] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:04 2022] DELETE /identity/v3/projects/a4b3543e331245afa5d9ca1520922082 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:05.091155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:05.103693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c81a37-cb30-402c-8c88-a7c63b1c086a service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:05.125388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:05.151230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:05.152695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:05.261604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:05.261604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.261604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.261604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] PATH_INFO: `/v3/users/761c1c81854545e1b2cbe40f32c4d725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.262104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=761c1c81854545e1b2cbe40f32c4d725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:05.262328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:05.264006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b03d24-b613-46ea-bcf8-1e46c47c825c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:05.272625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:05.279921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.279921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.279921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] PATH_INFO: `/v3/projects/47358a8232234c108bf4f74e6d72b6d0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.279921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47358a8232234c108bf4f74e6d72b6d0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:05.279921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:05.411314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761c1c81854545e1b2cbe40f32c4d725'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.411652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47358a8232234c108bf4f74e6d72b6d0', 'request_id': 'req-2558fbc8-20b8-4683-83a3-99763065e96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.412278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761c1c81854545e1b2cbe40f32c4d725'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.418453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47358a8232234c108bf4f74e6d72b6d0', 'request_id': 'req-2558fbc8-20b8-4683-83a3-99763065e96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.430635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761c1c81854545e1b2cbe40f32c4d725'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.484706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47358a8232234c108bf4f74e6d72b6d0', 'request_id': 'req-2558fbc8-20b8-4683-83a3-99763065e96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.496690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] The token cache is being invalidate because project 47358a8232234c108bf4f74e6d72b6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:05.497007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.497654 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2558fbc8-20b8-4683-83a3-99763065e96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.499159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1267/2552] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:05 2022] DELETE /identity/v3/projects/47358a8232234c108bf4f74e6d72b6d0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:05.503728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.504045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.504326 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.524322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:05.525525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:05.553549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761c1c81854545e1b2cbe40f32c4d725'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.562436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a041a2ac-70d8-4151-8884-15c9381d3145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761c1c81854545e1b2cbe40f32c4d725'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:05.574390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1286/2553] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:05 2022] DELETE /identity/v3/users/761c1c81854545e1b2cbe40f32c4d725 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:05.596544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:05.605267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1268/2554] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:05.608432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.608811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.609627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.634634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:05.635898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:05.696390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1287/2555] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:05.700673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:05.710933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0c136-e8e2-4100-afdb-df7be86a9ddd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:05.715536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:05.716199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.716484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.716758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.717431 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:05.717736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:05.719346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:05.732049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1269/2556] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:05.736019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:05.745362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d005d7-d762-4464-bf53-f3c825d1c694 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:05.750062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:05.750660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.750939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.751247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.751861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:05.753763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:05.879967 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1288/2557] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:05 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:05.881596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:05.892953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f26e97-696a-4f90-ab7e-984c36e48532 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:05.958171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:05.958820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:05.959608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:05.959919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:05.960544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:05.962460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.036783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.039075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.055631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:06.061227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f53ce-cadc-4f51-90b0-3abfa5f55220 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.066782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1270/2558] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:05 2022] POST /identity/v3/users => generated 279 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:06.067039 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.067450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.067735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.068038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] PATH_INFO: `/v3/projects/335fceab8fc44aa7b58bc0546d263306` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.070189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.071163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=335fceab8fc44aa7b58bc0546d263306)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.073120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.081832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748fc15-25f8-423b-98f1-e705ab0ba2fc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.087515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.088135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.088415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.088681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.089313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.095206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.102103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1271/2559] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:06.106052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.115688 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc87c4-166e-4d81-9273-908216d93fde admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.120415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.120979 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.124536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.124536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.124536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.124536 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.133884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1272/2560] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:06.137728 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.148587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b366f97-ed55-4478-aa72-821a7956252c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.154872 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.155505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.155832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.155832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] PATH_INFO: `/v3/projects/4e8d40930e7e4477856c92cba98caea2/users/92b702f3e4f24d61b5953ee101d3a52f/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.172078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8d40930e7e4477856c92cba98caea2, user_id=92b702f3e4f24d61b5953ee101d3a52f, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.174757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f080802-120a-4722-ac84-cdd4095e42e5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.246154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1273/2561] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:06 2022] PUT /identity/v3/projects/4e8d40930e7e4477856c92cba98caea2/users/92b702f3e4f24d61b5953ee101d3a52f/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:06.255030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752274d2-1f55-41f7-afbe-455f63355833 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.255030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752274d2-1f55-41f7-afbe-455f63355833 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.255030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752274d2-1f55-41f7-afbe-455f63355833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.255313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335fceab8fc44aa7b58bc0546d263306', 'request_id': 'req-014b1fef-fa04-445a-bf41-1cc149d63c7e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:06.260638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335fceab8fc44aa7b58bc0546d263306', 'request_id': 'req-014b1fef-fa04-445a-bf41-1cc149d63c7e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:06.267392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-752274d2-1f55-41f7-afbe-455f63355833 None None] MFA Rules not processed for user `2a9dbe3b7fdd482681b6ab94c26f4e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:06.268636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-752274d2-1f55-41f7-afbe-455f63355833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.322324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1274/2562] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:06.323547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.323856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.324178 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.337025 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335fceab8fc44aa7b58bc0546d263306', 'request_id': 'req-014b1fef-fa04-445a-bf41-1cc149d63c7e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:06.352527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] The token cache is being invalidate because project 335fceab8fc44aa7b58bc0546d263306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:06.352772 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:06.353894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-014b1fef-fa04-445a-bf41-1cc149d63c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:06.354319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] MFA Rules not processed for user `92b702f3e4f24d61b5953ee101d3a52f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:06.355596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1289/2563] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:06 2022] DELETE /identity/v3/projects/335fceab8fc44aa7b58bc0546d263306 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:06.355732 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.360775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e7c2-5e52-4593-856b-61f8b9f281f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.361065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e7c2-5e52-4593-856b-61f8b9f281f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.361340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9e7c2-5e52-4593-856b-61f8b9f281f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.383005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-13b9e7c2-5e52-4593-856b-61f8b9f281f9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:06.384247 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-13b9e7c2-5e52-4593-856b-61f8b9f281f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.386541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:06.448018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1275/2564] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:06.452535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.454283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.462413 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1290/2565] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:06.470444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.470778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.471063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.496649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:06.498002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:06.508450 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1291/2566] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:06.512877 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.524047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab8f33-e5c0-4539-a528-39fa13147969 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.531583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6d29-d239-4e4a-afa3-e893e3894dbd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.537820 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.538196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.538571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.538752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.539377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.540157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.541024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.541309 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.542372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.543551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.544640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:06.546212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.549769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.573881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1292/2567] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:06.575060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.596461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a2685-1e3e-4747-943b-ddb999c2ad2d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.608027 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.608841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.609296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.609816 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.610588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.612681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.626279 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1276/2568] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/projects => generated 355 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:06.630933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.711005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430ee74-4605-488c-97ba-fb3dd44be383 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.715751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.716391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.716678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.716951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.717572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.719504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.728083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1293/2569] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:06.732267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.784895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:06.802023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8cd04-7291-4ee6-a643-c59821013601 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.808384 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.809005 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.809295 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.809614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.810196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.812338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.824129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1277/2570] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:06.826705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.837958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00489d7-e76d-4d7e-a78a-986591ed3d49 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.842877 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.843583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.843838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.844110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.844927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.846508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.858011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1278/2571] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:06.862179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.872757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c6507-9473-45c4-b549-ed341a31ad80 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.877351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.877997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.878277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.878549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] PATH_INFO: `/v3/projects/01552e033daa41dca9748bcaf1861790/users/25bd03c957af4921af7ca06d2fba166c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.890642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:06.894364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01552e033daa41dca9748bcaf1861790, user_id=25bd03c957af4921af7ca06d2fba166c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.897586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.917851 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1294/2572] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:06 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:06.918942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.928621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44082b07-a690-4251-a13c-ca492b1a04a0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.933229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.933807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.934083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.934353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.934975 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.936582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.948067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1295/2573] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:06.953677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.964214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d211471-f9ad-4d21-8b3b-b771914d4a82 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:06.968893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:06.969516 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:06.969862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:06.975142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:06.975142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:06.975142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:06.980962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1279/2574] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:06 2022] PUT /identity/v3/projects/01552e033daa41dca9748bcaf1861790/users/25bd03c957af4921af7ca06d2fba166c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:06.985533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:06.990300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1296/2575] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:06.991852 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.003015 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a1e62-9ba2-4199-88e7-a96d205b7cbc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.053849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b04047-daed-48b1-b515-4b558e509728 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.058694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.059383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.059844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.060195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.061134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.062997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.070456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.071119 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.071575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.071698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] PATH_INFO: `/v3/projects/fb721f8294fa43119dfaf546b58091b7/users/f7886eff6ce848dbbb3d0b6ef669bfcc/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.075274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1280/2576] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:07.080778 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.090390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb721f8294fa43119dfaf546b58091b7, user_id=f7886eff6ce848dbbb3d0b6ef669bfcc, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.093525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b68b5-d719-46ed-9574-17cd764fb09a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.100459 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d3b59-ae65-4f69-a169-846ccc0de9de admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.100676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.101336 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.101606 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.101896 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] PATH_INFO: `/v3/projects/01552e033daa41dca9748bcaf1861790/users/25bd03c957af4921af7ca06d2fba166c/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.106137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01552e033daa41dca9748bcaf1861790, user_id=25bd03c957af4921af7ca06d2fba166c, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.108952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072b22c-ebef-4da4-91ee-1a392caa3890 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.123549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1281/2577] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:07 2022] PUT /identity/v3/projects/01552e033daa41dca9748bcaf1861790/users/25bd03c957af4921af7ca06d2fba166c/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:07.134574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.134859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.135140 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.166426 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] MFA Rules not processed for user `25bd03c957af4921af7ca06d2fba166c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:07.167656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:07.207351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1297/2578] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:06 2022] PUT /identity/v3/projects/fb721f8294fa43119dfaf546b58091b7/users/f7886eff6ce848dbbb3d0b6ef669bfcc/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:07.223166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.223789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.225083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.248547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1282/2579] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:07.261003 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.263670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:07.264769 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] MFA Rules not processed for user `f7886eff6ce848dbbb3d0b6ef669bfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:07.265960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:07.324478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314c4ee-fb9a-47a8-b22e-2d5b0f5e7299 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.337524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:07.340942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1298/2580] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:07.345622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.383337 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:07.397780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.399011 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.400153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.400153 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.427623 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449bf7d-e121-4a7d-92fe-6b79ff268d9e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.431655 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.432299 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.432641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.432916 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.433536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.435528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.465596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.467113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.477069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1283/2581] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:07.528572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1299/2582] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:07 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:07.532301 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.543973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65196ff-ec22-4478-b12f-5ffbc59dee12 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.649245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.649884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.650743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.651023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.651664 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.659572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.704893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.706690 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:07.766773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4e0ff-233a-4a65-a94d-9596830fcc3b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.770545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.771167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.771446 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.771717 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.790170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1284/2583] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:07 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:07.793955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.804227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585bf35-a7ae-4072-86d5-a7c8ba76d7bd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.808819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.809395 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.809701 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.809978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.810596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.812197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.822734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1285/2584] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:07.826839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:07.831343 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.832920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.837133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49782-d97b-484f-875c-160e01bce607 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:07.842143 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:07.842632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1300/2585] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:07.842932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:07.843245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:07.843518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] PATH_INFO: `/v3/projects/6ad17ff103e6446195370483bfda9090/users/f787c48e5da147d69019e5dd0099e3ac/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:07.858969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad17ff103e6446195370483bfda9090, user_id=f787c48e5da147d69019e5dd0099e3ac, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:07.861645 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:07.972390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1286/2586] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:07 2022] PUT /identity/v3/projects/6ad17ff103e6446195370483bfda9090/users/f787c48e5da147d69019e5dd0099e3ac/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:07.977835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:08.010698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfad409-23fb-4c34-a330-ae2e55f5f231 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:08.054471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:08.055171 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:08.055556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:08.055865 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:08.056634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:08.058339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d42c41-e27e-4683-87da-4fd2fbd121dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:08.068848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1301/2587] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:08.073488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:08.085837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e720fd5-9066-41c5-9f5f-f1aeb71a6887 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:08.090689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:08.091402 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:08.091684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:08.091990 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] PATH_INFO: `/v3/projects/6ad17ff103e6446195370483bfda9090/users/f787c48e5da147d69019e5dd0099e3ac/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:08.093951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad17ff103e6446195370483bfda9090, user_id=f787c48e5da147d69019e5dd0099e3ac, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:08.097129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:08.110248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1287/2588] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:08 2022] PUT /identity/v3/projects/6ad17ff103e6446195370483bfda9090/users/f787c48e5da147d69019e5dd0099e3ac/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:08.124646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:08.124937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:08.125279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:08.133339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:08.155369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] MFA Rules not processed for user `f787c48e5da147d69019e5dd0099e3ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:08.156610 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:08.233210 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1302/2589] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:08.245287 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:08.254159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:08.304528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876202b-62c1-4dfa-b989-43cd64324330 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:08.308182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:08.308774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:08.309046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:08.309313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:08.318777 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:08.320225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10ae51-0b67-42bc-991a-59247d2cc563 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:08.329390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1288/2590] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:09.377320 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:09.440445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539af18-f8d9-48a4-b36c-a1b73a909bb6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:09.444948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:09.445572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:09.445850 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:09.446128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] PATH_INFO: `/v3/users/f7886eff6ce848dbbb3d0b6ef669bfcc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:09.447256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7886eff6ce848dbbb3d0b6ef669bfcc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:09.449225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:09.686634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7886eff6ce848dbbb3d0b6ef669bfcc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:09.687280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7886eff6ce848dbbb3d0b6ef669bfcc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:09.700696 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7886eff6ce848dbbb3d0b6ef669bfcc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:09.788438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7886eff6ce848dbbb3d0b6ef669bfcc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:09.796463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7886eff6ce848dbbb3d0b6ef669bfcc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:09.803161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1303/2591] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:09 2022] DELETE /identity/v3/users/f7886eff6ce848dbbb3d0b6ef669bfcc => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:09.807476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:09.807973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:09.808084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:09.827373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:09.828591 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:09.917015 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1289/2592] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:09.925390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:09.926844 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:09.937698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:09.984111 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68b53e-c6ce-4707-a5a2-23b97412572b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:09.987677 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:09.988306 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:09.988583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:09.988854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:09.998226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:10.008827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:10.009995 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1304/2593] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:10.693368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:10.694900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:10.718620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a67c4-0dad-4c94-ae83-e14f1dead554 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:10.725052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:10.725751 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:10.726098 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:10.726376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] PATH_INFO: `/v3/projects/fb721f8294fa43119dfaf546b58091b7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:10.727488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb721f8294fa43119dfaf546b58091b7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:10.729457 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:10.882804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb721f8294fa43119dfaf546b58091b7', 'request_id': 'req-6237802a-49fe-4b13-a0ff-0a0448a09a84'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:10.889082 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb721f8294fa43119dfaf546b58091b7', 'request_id': 'req-6237802a-49fe-4b13-a0ff-0a0448a09a84'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:10.988806 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb721f8294fa43119dfaf546b58091b7', 'request_id': 'req-6237802a-49fe-4b13-a0ff-0a0448a09a84'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.003662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] The token cache is being invalidate because project fb721f8294fa43119dfaf546b58091b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:11.004188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.004974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.007088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1290/2594] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:10 2022] DELETE /identity/v3/projects/fb721f8294fa43119dfaf546b58091b7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:11.011386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:11.022200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079dc54-652f-4612-9864-882b0d733ee4 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:11.086055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:11.086680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:11.086953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:11.087221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] PATH_INFO: `/v3/users/f787c48e5da147d69019e5dd0099e3ac` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:11.088339 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f787c48e5da147d69019e5dd0099e3ac)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:11.090311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:11.394243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f787c48e5da147d69019e5dd0099e3ac'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.395067 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f787c48e5da147d69019e5dd0099e3ac'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.408132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f787c48e5da147d69019e5dd0099e3ac'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.492286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f787c48e5da147d69019e5dd0099e3ac'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.500954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c13a4352-4c0f-4de9-bfe6-fc387b9a1466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f787c48e5da147d69019e5dd0099e3ac'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:11.507176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1305/2595] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:11 2022] DELETE /identity/v3/users/f787c48e5da147d69019e5dd0099e3ac => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:12.063851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.065366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:12.080117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237802a-49fe-4b13-a0ff-0a0448a09a84 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.114470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:12.143730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.144387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.144667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.144939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] PATH_INFO: `/v3/projects/6ad17ff103e6446195370483bfda9090` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.146821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ad17ff103e6446195370483bfda9090)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.148757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.259097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ad17ff103e6446195370483bfda9090', 'request_id': 'req-4dfdfc5e-61d0-411d-9417-512a89fbb7be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:12.265769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ad17ff103e6446195370483bfda9090', 'request_id': 'req-4dfdfc5e-61d0-411d-9417-512a89fbb7be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:12.320509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ad17ff103e6446195370483bfda9090', 'request_id': 'req-4dfdfc5e-61d0-411d-9417-512a89fbb7be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:12.331503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] The token cache is being invalidate because project 6ad17ff103e6446195370483bfda9090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:12.331829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:12.332477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4dfdfc5e-61d0-411d-9417-512a89fbb7be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:12.333963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1291/2596] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:12 2022] DELETE /identity/v3/projects/6ad17ff103e6446195370483bfda9090 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:12.339063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.339589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.339882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.361918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:12.363149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:12.395858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:12.437603 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1306/2597] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:12.446227 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.446516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.446790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.466166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:12.467364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:12.475422 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1292/2598] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:12.482805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.493135 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65b961-a2a6-4ce4-a1de-869e39512aa9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.498008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.498679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.499054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.499350 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.500059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:12.500360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.501973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.514285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1307/2599] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:12.518211 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.527389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ba2e8-30d5-4caf-a924-7cbca78b027e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.532419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.533084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.533430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.533769 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.534458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.537310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.616570 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1293/2600] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:12 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:12.620640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.633114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229edf8-1efe-4613-934d-721529805a07 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.706519 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.706966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.707443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.707794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.708999 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.710927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.819604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1308/2601] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:12 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:12.819711 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.826358 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af466dfb-8072-4280-9b39-34fe56ae10d6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.831159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.831889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.832237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.832584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.833359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.835018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.863061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1294/2602] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:12.868224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.882308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7afdb-8882-4575-879d-37ec62c0ccdc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.887002 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.887628 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.887951 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.888212 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.889029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.890599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da551-ee7d-4919-8eea-8278b43b7af6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:12.920824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1309/2603] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:12.922634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:12.934429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff957c-2653-4262-b53b-25f6663765f5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:12.941953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:12.942775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:12.943148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:12.943605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] PATH_INFO: `/v3/projects/81e67c17eaf3498186e999d521b1c67a/users/e68621a299644cba98dba6ec3bdc1e2a/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:12.963325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e67c17eaf3498186e999d521b1c67a, user_id=e68621a299644cba98dba6ec3bdc1e2a, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:12.966440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:13.051885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1295/2604] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:12 2022] PUT /identity/v3/projects/81e67c17eaf3498186e999d521b1c67a/users/e68621a299644cba98dba6ec3bdc1e2a/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:13.056266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:13.056553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:13.056825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:13.103988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] MFA Rules not processed for user `e68621a299644cba98dba6ec3bdc1e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:13.105614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:13.183676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1310/2605] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:13 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:13.189056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:13.274875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f39a7-9300-4c7d-bdae-4038aa7bd19d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:13.282451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:13.283995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:13.284513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:13.284794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:13.285756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:13.288366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:13.303430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:13.430198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1296/2606] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:13 2022] POST /identity/v3/projects => generated 401 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:13.438074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:13.462583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330e835-16de-4d9a-a852-2406479ee449 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:13.490266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:13.538554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:13.539437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:13.539709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:13.540604 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:13.540722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:13.543241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:13.675961 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1311/2607] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:13 2022] POST /identity/v3/users => generated 307 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:13.681715 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:13.700596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746fab3-6d7b-484e-afd9-22de2b14753a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:13.714997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:13.717014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:13.717633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:13.720118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:13.721045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:13.723308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:13.764087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1297/2608] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:13.765853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:13.782293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df287c1e-7b73-4552-affa-5a0039a8721b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:13.796804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:13.797997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:13.798489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:13.798959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] PATH_INFO: `/v3/projects/c793fb0b03d24b7a87a7da9a458e015f/users/abf8683d176b48abbe337427b69664cf/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:13.842837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c793fb0b03d24b7a87a7da9a458e015f, user_id=abf8683d176b48abbe337427b69664cf, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:13.849778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:14.024612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:14.026425 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1312/2609] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:13 2022] PUT /identity/v3/projects/c793fb0b03d24b7a87a7da9a458e015f/users/abf8683d176b48abbe337427b69664cf/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:14.036488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15166db-c714-48b3-aed4-d207671f93a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:14.104416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:14.105496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:14.105881 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:14.106259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:14.107343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:14.109453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d643e6-167a-43c0-b58e-32e5189481ee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:14.128311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:14.131091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1298/2610] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:14.140894 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82df5b-4688-4bab-a9a5-6dd89ef5099f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:14.146369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:14.147325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:14.149870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:14.149870 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] PATH_INFO: `/v3/projects/c793fb0b03d24b7a87a7da9a458e015f/users/abf8683d176b48abbe337427b69664cf/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:14.150186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c793fb0b03d24b7a87a7da9a458e015f, user_id=abf8683d176b48abbe337427b69664cf, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:14.153092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:14.165528 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1313/2611] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:14 2022] PUT /identity/v3/projects/c793fb0b03d24b7a87a7da9a458e015f/users/abf8683d176b48abbe337427b69664cf/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:14.176079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:14.176373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:14.176677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:14.207485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] MFA Rules not processed for user `abf8683d176b48abbe337427b69664cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:14.209434 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:14.286420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1299/2612] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:14 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:14.298926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:14.300775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:14.366678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22d413-46f4-4a3c-968b-fbb595475bd1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:14.370617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:14.371235 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:14.371513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:14.371826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:14.384520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:14.386562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e98e7f-e40a-4a5a-9f4c-025c3d8461b5 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:14.397466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1314/2613] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:14.659737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:14.671243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b67cdf-1e70-49d6-a155-b4966a8f3e38 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:14.675496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:14.676237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:14.676488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:14.677284 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:14.745043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:14.747649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:14.749203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:14.761398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1300/2614] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:18.937537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b5d5-cfba-45ec-a39a-2ea9143c0a67 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:18.938021 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b5d5-cfba-45ec-a39a-2ea9143c0a67 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:18.938123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b5d5-cfba-45ec-a39a-2ea9143c0a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:18.958248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4fd5b5d5-cfba-45ec-a39a-2ea9143c0a67 None None] MFA Rules not processed for user `08d81bcd5a4f43ce8b4839ee15cb163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:18.960122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4fd5b5d5-cfba-45ec-a39a-2ea9143c0a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:19.016296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1315/2615] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:20.086531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:20.149165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5fd6c-f084-4b84-b081-170191083ca1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:20.153939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:20.154584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:20.154862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:20.155134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] PATH_INFO: `/v3/users/e68621a299644cba98dba6ec3bdc1e2a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:20.156325 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68621a299644cba98dba6ec3bdc1e2a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:20.158286 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:20.389544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68621a299644cba98dba6ec3bdc1e2a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:20.390132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68621a299644cba98dba6ec3bdc1e2a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:20.396954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68621a299644cba98dba6ec3bdc1e2a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:20.410415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68621a299644cba98dba6ec3bdc1e2a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:20.478835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68621a299644cba98dba6ec3bdc1e2a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:20.485219 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1301/2616] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:20 2022] DELETE /identity/v3/users/e68621a299644cba98dba6ec3bdc1e2a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:20.489659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:20.490148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:20.490261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:20.508609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:20.509807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:20.528287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:20.569978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1316/2617] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:20.580209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:20.582718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:20.652719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16f01-575f-4b19-8bab-f69503b810f3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:20.656641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:20.657328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:20.657681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:20.658206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:20.674970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:20.676795 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:20.698915 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1302/2618] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:21.195791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:21.195791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:21.203544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d134-5954-46b9-b718-1b04a2c09c0c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:21.208612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:21.209621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:21.209905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:21.210181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] PATH_INFO: `/v3/projects/81e67c17eaf3498186e999d521b1c67a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:21.212418 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e67c17eaf3498186e999d521b1c67a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:21.214494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:21.359162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e67c17eaf3498186e999d521b1c67a', 'request_id': 'req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.365256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e67c17eaf3498186e999d521b1c67a', 'request_id': 'req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.496187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e67c17eaf3498186e999d521b1c67a', 'request_id': 'req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.507037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] The token cache is being invalidate because project 81e67c17eaf3498186e999d521b1c67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:21.507324 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.507974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.509467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1317/2619] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:21 2022] DELETE /identity/v3/projects/81e67c17eaf3498186e999d521b1c67a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:21.513537 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:21.523237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a688d6-ae96-421f-8e37-f45dd7b5b7e6 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:21.535321 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:21.581951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:21.582576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:21.582888 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:21.583170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] PATH_INFO: `/v3/users/abf8683d176b48abbe337427b69664cf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:21.584353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf8683d176b48abbe337427b69664cf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:21.586339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:21.791479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf8683d176b48abbe337427b69664cf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.793158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf8683d176b48abbe337427b69664cf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.801485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf8683d176b48abbe337427b69664cf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.816318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf8683d176b48abbe337427b69664cf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.907174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d024f2ce-8f01-40e6-b487-47fdd8dc2e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf8683d176b48abbe337427b69664cf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:21.917587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1303/2620] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:21 2022] DELETE /identity/v3/users/abf8683d176b48abbe337427b69664cf => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:22.434521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:22.436068 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:22.455205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a88f6-7fa2-4b8b-9d62-65b10cbb75e3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:22.525253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:22.525923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:22.526165 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:22.526445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] PATH_INFO: `/v3/projects/c793fb0b03d24b7a87a7da9a458e015f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:22.527581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c793fb0b03d24b7a87a7da9a458e015f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:22.529649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:22.668680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c793fb0b03d24b7a87a7da9a458e015f', 'request_id': 'req-78569212-6d15-4b0d-898f-c2ba49a089dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:22.669792 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:22.675489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c793fb0b03d24b7a87a7da9a458e015f', 'request_id': 'req-78569212-6d15-4b0d-898f-c2ba49a089dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:22.787372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c793fb0b03d24b7a87a7da9a458e015f', 'request_id': 'req-78569212-6d15-4b0d-898f-c2ba49a089dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:22.799281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] The token cache is being invalidate because project c793fb0b03d24b7a87a7da9a458e015f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:22.799577 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:22.800373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-78569212-6d15-4b0d-898f-c2ba49a089dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:22.802375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1318/2621] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:22 2022] DELETE /identity/v3/projects/c793fb0b03d24b7a87a7da9a458e015f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:22.806759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:22.807243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:22.807345 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:22.826658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:22.827907 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:22.902290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1304/2622] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:22.911474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:22.911810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:22.912098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:22.942472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:22.944647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:22.963646 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1319/2623] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:22.967824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:22.986053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809044dd-5719-4217-8ce3-0eaaa1c506ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:22.995056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:22.995719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:22.996170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:22.996447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:22.997246 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:23.001507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.001507 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.018465 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1305/2624] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:23.019361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.034454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c9d1-679a-4cb5-bdcb-933f25e42d2c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.040969 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.041823 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.042317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.042996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.043862 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.045903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.151565 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1320/2625] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:23 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:23.156149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.176411 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae6a07-7d7b-4791-b3c0-21ef9a6035d5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.210084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:23.262090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.262805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.263587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.263587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.264195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.266160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.416975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1306/2626] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:23 2022] POST /identity/v3/users => generated 288 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:23.425509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.435529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d3210-441a-4d1e-a0cd-4b2d12b2b25b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.439989 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.440576 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.440849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.441116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.441748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.443316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.453227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1321/2627] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:23.464632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.474496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c98f4b-f2f8-4a45-b88d-ddd504e1de9e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.479724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.480334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.480621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.481565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.482208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.484007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff775c-8e79-4f58-ba6c-6cec973fa83c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.494997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1307/2628] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:23.499133 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.511957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d29f8-cd95-49df-9d8a-d81066dc3462 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.517845 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.518515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.518873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.519157 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] PATH_INFO: `/v3/projects/3ceb1ac7a84049f2b5264ca3734729ba/users/72d4ff45bee9499ebdc92e84a1331571/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.542308 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ceb1ac7a84049f2b5264ca3734729ba, user_id=72d4ff45bee9499ebdc92e84a1331571, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.546738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.679009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1322/2629] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:23 2022] PUT /identity/v3/projects/3ceb1ac7a84049f2b5264ca3734729ba/users/72d4ff45bee9499ebdc92e84a1331571/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:23.680824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.681123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.681467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.716970 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] MFA Rules not processed for user `72d4ff45bee9499ebdc92e84a1331571`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:23.719782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:23.787510 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1308/2630] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:23.808777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:23.810400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:23.873206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a11f21-406b-4da0-acd5-6043fca108c5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:23.876974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:23.877637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:23.877976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:23.878175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:23.890797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:23.892277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39189040-0011-48df-9c0c-3f4f3b06fa00 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:23.902755 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1323/2631] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:27.669453 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:27.670869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:27.735542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847fe36-57b9-419c-bfd8-35b35452b5b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:27.739920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:27.740575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:27.740855 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:27.741128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] PATH_INFO: `/v3/users/92b702f3e4f24d61b5953ee101d3a52f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:27.742236 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b702f3e4f24d61b5953ee101d3a52f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:27.744185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:27.913705 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:27.925202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b702f3e4f24d61b5953ee101d3a52f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:27.925862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b702f3e4f24d61b5953ee101d3a52f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:27.933056 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b702f3e4f24d61b5953ee101d3a52f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:27.945355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b702f3e4f24d61b5953ee101d3a52f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.015360 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b702f3e4f24d61b5953ee101d3a52f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.021398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1309/2632] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:27 2022] DELETE /identity/v3/users/92b702f3e4f24d61b5953ee101d3a52f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:28.025927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:28.026401 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:28.026532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:28.045134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:28.046405 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:28.102533 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1324/2633] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:28.112404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:28.114260 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:28.170887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088541-d711-4634-ab09-e8992cb354ad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:28.174614 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:28.175322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:28.175704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:28.177257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:28.194065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:28.194065 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:28.203900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1310/2634] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:28.623076 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:28.624406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:28.636534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c7fe6-ab42-4c41-bd99-45dbbf6406e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:28.641217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:28.641857 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:28.642271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:28.642543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] PATH_INFO: `/v3/projects/4e8d40930e7e4477856c92cba98caea2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:28.643701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8d40930e7e4477856c92cba98caea2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:28.645657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:28.658126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:28.805721 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8d40930e7e4477856c92cba98caea2', 'request_id': 'req-0dd77a03-ee54-40bf-9cd6-ddd952876798'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.811782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8d40930e7e4477856c92cba98caea2', 'request_id': 'req-0dd77a03-ee54-40bf-9cd6-ddd952876798'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.882911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8d40930e7e4477856c92cba98caea2', 'request_id': 'req-0dd77a03-ee54-40bf-9cd6-ddd952876798'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.894033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] The token cache is being invalidate because project 4e8d40930e7e4477856c92cba98caea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:28.894327 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.894958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:28.896509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1325/2635] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:28 2022] DELETE /identity/v3/projects/4e8d40930e7e4477856c92cba98caea2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:28.900503 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:28.910743 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa2f55-83aa-49b7-95d0-7785bed0edb8 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:28.966201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:28.966815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:28.967095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:28.967370 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] PATH_INFO: `/v3/users/25bd03c957af4921af7ca06d2fba166c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:28.968509 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bd03c957af4921af7ca06d2fba166c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:28.970408 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:29.160275 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bd03c957af4921af7ca06d2fba166c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.160941 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bd03c957af4921af7ca06d2fba166c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.166789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:29.168892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bd03c957af4921af7ca06d2fba166c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.182291 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bd03c957af4921af7ca06d2fba166c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.255134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0a23b459-8d44-4763-ac44-03fe12ca7db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bd03c957af4921af7ca06d2fba166c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.261368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1311/2636] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:28 2022] DELETE /identity/v3/users/25bd03c957af4921af7ca06d2fba166c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:29.656435 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:29.658192 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:29.675342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd77a03-ee54-40bf-9cd6-ddd952876798 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:29.740470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:29.741123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:29.741400 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:29.741671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] PATH_INFO: `/v3/projects/01552e033daa41dca9748bcaf1861790` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:29.742783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01552e033daa41dca9748bcaf1861790)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:29.744726 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:29.849552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01552e033daa41dca9748bcaf1861790', 'request_id': 'req-025e2d05-de95-453e-afa2-7345ccf97697'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.855529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01552e033daa41dca9748bcaf1861790', 'request_id': 'req-025e2d05-de95-453e-afa2-7345ccf97697'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.918455 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01552e033daa41dca9748bcaf1861790', 'request_id': 'req-025e2d05-de95-453e-afa2-7345ccf97697'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.929724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] The token cache is being invalidate because project 01552e033daa41dca9748bcaf1861790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:29.930014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.930663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-025e2d05-de95-453e-afa2-7345ccf97697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:29.932196 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1326/2637] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:29 2022] DELETE /identity/v3/projects/01552e033daa41dca9748bcaf1861790 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:29.937218 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:29.937692 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:29.937817 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:29.957517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:29.958699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:30.023022 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1312/2638] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:30.030700 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.031086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.031433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.051586 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:30.052837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:30.061294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1327/2639] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:30.065541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.075783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba2a8d-529f-414e-adc5-7a4539d15419 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.080332 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.080916 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.081205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.081454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.082086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:30.082442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.084035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.095256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1313/2640] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:30.099478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.108867 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc1510-4ed1-4b72-b6f0-8d4fea903d6f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.113554 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.114145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.114422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.114707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.115314 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.117735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.200669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1328/2641] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:30.204623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.214552 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3de3-53bf-42c7-b422-d7501c6f1df0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.269797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.270385 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.270695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.270974 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.271634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.273638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.375138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1314/2642] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/users => generated 282 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:30.376796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.382815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:30.388022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636829-c164-4855-b254-4971379e8ed0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.392588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.393209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.393484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.393752 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.394532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.396182 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.406145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1329/2643] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:30.410618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.423949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338539-1635-4511-843d-2a8bee1a5994 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.433699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.433699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.433699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.433699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.434147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.434776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43902b-cf32-4273-88b4-a1828381fc90 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.449649 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1315/2644] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:30.451174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.463581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b9d7-981d-4477-8bd1-5b3708888eb6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.467222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.467988 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.468332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.468673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] PATH_INFO: `/v3/projects/53aa10d4b64e4103b429817e441ed38d/users/10c6bae9f76f42a9bda9fab6a78e0e3b/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.483612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53aa10d4b64e4103b429817e441ed38d, user_id=10c6bae9f76f42a9bda9fab6a78e0e3b, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.486245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.609976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1330/2645] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:30 2022] PUT /identity/v3/projects/53aa10d4b64e4103b429817e441ed38d/users/10c6bae9f76f42a9bda9fab6a78e0e3b/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:30.621901 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.622190 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.622463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.650123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] MFA Rules not processed for user `10c6bae9f76f42a9bda9fab6a78e0e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:30.651351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:30.677602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:30.735927 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1316/2646] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:30.740121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.814679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdfd8-b738-4958-b9e7-e7e042d5af19 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:30.819611 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:30.820331 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:30.820684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:30.821022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:30.821756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:30.823837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:30.949800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1331/2647] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:30.952757 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:30.964188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1236159-6cd9-4ef1-8ae1-4ffccd346fd2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.020839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.021448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.021726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.021996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.023150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.025199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.176110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1317/2648] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:30 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:31.180123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:31.190473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7992e-8efe-4aaa-8183-247fc26e0f81 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.197412 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.198225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.198582 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.198861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.199632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.202272 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.214389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1332/2649] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:31.218873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:31.228702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea49f2-5b66-4311-99a2-7e7ed6a0acf7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.233184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.233810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.234084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.234354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] PATH_INFO: `/v3/projects/8a972b2e98004c3b95c3e385dd836301/users/8cdf9706e9174742831b7feb46635a7f/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.248746 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a972b2e98004c3b95c3e385dd836301, user_id=8cdf9706e9174742831b7feb46635a7f, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.251353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.323617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1318/2650] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:31 2022] PUT /identity/v3/projects/8a972b2e98004c3b95c3e385dd836301/users/8cdf9706e9174742831b7feb46635a7f/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:31.327484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:31.338109 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cff7b-192e-4a39-a1e3-7426861ad946 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.394908 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.395608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.395927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.396200 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.396841 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.399049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352eb777-7317-42f7-8b31-11cbe77311bf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.409080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1333/2651] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:31.413497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:31.423589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e217f9-51e0-4d00-85e4-687c918125d3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.428229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.431641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.431641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.431641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] PATH_INFO: `/v3/projects/8a972b2e98004c3b95c3e385dd836301/users/8cdf9706e9174742831b7feb46635a7f/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.431641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a972b2e98004c3b95c3e385dd836301, user_id=8cdf9706e9174742831b7feb46635a7f, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.434152 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.445703 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1319/2652] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:31 2022] PUT /identity/v3/projects/8a972b2e98004c3b95c3e385dd836301/users/8cdf9706e9174742831b7feb46635a7f/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:31.455853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.456225 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.456504 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.486284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] MFA Rules not processed for user `8cdf9706e9174742831b7feb46635a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:31.487490 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:31.643139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:31.686880 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1334/2653] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:31.701248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:31.702925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:31.760522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea349b7d-9955-4d69-b6c0-38de98b274ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:31.764062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:31.765261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:31.765534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:31.765802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:31.777371 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:31.779147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:31.789568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1320/2654] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:31.997095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:32.007098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ae1fa-5211-4536-b1ec-677ab7d637e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:32.010861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:32.011555 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:32.011923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:32.012262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:32.090662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:32.092583 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296efcbe-d730-430f-b3d5-eae631c23a19 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:32.102497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1335/2655] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:41.901484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:42.592453 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=45 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:48.539563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:48.600195 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0a186-01db-466b-a629-43e30addf276 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:48.605533 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:48.606117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:48.606354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:48.606630 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] PATH_INFO: `/v3/users/10c6bae9f76f42a9bda9fab6a78e0e3b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:48.607829 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c6bae9f76f42a9bda9fab6a78e0e3b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:48.609789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:48.793214 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c6bae9f76f42a9bda9fab6a78e0e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:48.793819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c6bae9f76f42a9bda9fab6a78e0e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:48.801455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c6bae9f76f42a9bda9fab6a78e0e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:48.815191 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c6bae9f76f42a9bda9fab6a78e0e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:48.910636 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c6bae9f76f42a9bda9fab6a78e0e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:48.917828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1321/2656] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:48 2022] DELETE /identity/v3/users/10c6bae9f76f42a9bda9fab6a78e0e3b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:48.922298 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:48.922820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:48.922820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:48.941162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:48.942355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:49.000150 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1336/2657] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:49.010185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:49.011656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:49.072818 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8314717-12d4-46fc-9226-b354c3c9a39e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:49.076754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:49.077470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:49.077661 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:49.077930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:49.088627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:49.090041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:49.099856 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1322/2658] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:49.541493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:49.542977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:49.555297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcc45f-d79c-480e-ab48-d3a922b7463b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:49.560084 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:49.560713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:49.560982 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:49.561246 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] PATH_INFO: `/v3/projects/53aa10d4b64e4103b429817e441ed38d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:49.562349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53aa10d4b64e4103b429817e441ed38d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:49.564251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:49.659776 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53aa10d4b64e4103b429817e441ed38d', 'request_id': 'req-51275b86-bec2-475b-9617-76a92f8b831f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:49.665947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53aa10d4b64e4103b429817e441ed38d', 'request_id': 'req-51275b86-bec2-475b-9617-76a92f8b831f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:49.757569 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53aa10d4b64e4103b429817e441ed38d', 'request_id': 'req-51275b86-bec2-475b-9617-76a92f8b831f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:49.769335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] The token cache is being invalidate because project 53aa10d4b64e4103b429817e441ed38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:49.769622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:49.770259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:49.771722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1337/2659] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:49 2022] DELETE /identity/v3/projects/53aa10d4b64e4103b429817e441ed38d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:49.775840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:49.786171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859a753-f974-4ebf-ade7-1739e6a863db service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:49.805479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:49.844930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:49.845559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:49.845839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:49.846110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] PATH_INFO: `/v3/users/8cdf9706e9174742831b7feb46635a7f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:49.847519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cdf9706e9174742831b7feb46635a7f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:49.849493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:50.163815 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cdf9706e9174742831b7feb46635a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.164398 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cdf9706e9174742831b7feb46635a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.171131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cdf9706e9174742831b7feb46635a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.183918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cdf9706e9174742831b7feb46635a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.288603 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cdf9706e9174742831b7feb46635a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.294926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1323/2660] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:49 2022] DELETE /identity/v3/users/8cdf9706e9174742831b7feb46635a7f => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:50.571825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:50.575043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:50.589187 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51275b86-bec2-475b-9617-76a92f8b831f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:50.651466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:50.652668 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:50.653132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:50.653398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] PATH_INFO: `/v3/users/72d4ff45bee9499ebdc92e84a1331571` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:50.654552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d4ff45bee9499ebdc92e84a1331571)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:50.656474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:50.781658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:50.783126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:50.796249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f78ba3-921d-4624-a4c9-5e574d85285d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:50.800963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:50.801920 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:50.802202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:50.802479 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] PATH_INFO: `/v3/projects/8a972b2e98004c3b95c3e385dd836301` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:50.803695 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a972b2e98004c3b95c3e385dd836301)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:50.805794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:50.809019 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:50.913797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d4ff45bee9499ebdc92e84a1331571'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.914463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d4ff45bee9499ebdc92e84a1331571'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.918783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a972b2e98004c3b95c3e385dd836301', 'request_id': 'req-65b0d235-40a5-426e-b21e-10c14dc360b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.925168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a972b2e98004c3b95c3e385dd836301', 'request_id': 'req-65b0d235-40a5-426e-b21e-10c14dc360b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:50.934047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d4ff45bee9499ebdc92e84a1331571'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.013584 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a972b2e98004c3b95c3e385dd836301', 'request_id': 'req-65b0d235-40a5-426e-b21e-10c14dc360b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.025694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] The token cache is being invalidate because project 8a972b2e98004c3b95c3e385dd836301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:51.025694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.025694 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65b0d235-40a5-426e-b21e-10c14dc360b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.027182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1324/2661] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:50 2022] DELETE /identity/v3/projects/8a972b2e98004c3b95c3e385dd836301 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:51.033104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b715-5973-46e9-a707-9178931f27a7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.033846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b715-5973-46e9-a707-9178931f27a7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.034335 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b715-5973-46e9-a707-9178931f27a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.059553 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7f25b715-5973-46e9-a707-9178931f27a7 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:51.063950 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7f25b715-5973-46e9-a707-9178931f27a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.080245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:51.125581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1325/2662] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:51.133991 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d4ff45bee9499ebdc92e84a1331571'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.134921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.135210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.135514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.141934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a3b388d-2978-4f69-81b3-f3843b7eacc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d4ff45bee9499ebdc92e84a1331571'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.147915 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1338/2663] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:12:50 2022] DELETE /identity/v3/users/72d4ff45bee9499ebdc92e84a1331571 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:51.152842 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.153127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.153399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.155188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:51.156410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.171536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:51.172777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.218401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1326/2664] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:51.219311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.227914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafca4d-db36-45d1-ae2c-1c9017218109 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.231097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1339/2665] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:51.232516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.233095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.233373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.233644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.234280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:51.234577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.236217 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.241514 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.242839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.248547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1327/2666] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 11:12:51.252827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.264451 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339157a-7dcd-47fe-8598-f4a06d90916c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.272166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.273177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.273669 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.273955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.274842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.276823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.306627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e865ea-97ef-46cc-be01-3ba3e80e1c36 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.310287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.310887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.311175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.311428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.320683 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.322199 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.331471 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1340/2667] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:51.420157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1328/2668] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:51.421281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.431843 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48468c2b-f26f-4be2-9113-8d357d9e445e service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.491942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:51.504911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.505517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.505791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.506055 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.506909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.509016 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.617102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1341/2669] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:51.621495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.635026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b407-6886-4822-8002-13896f17107c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.641077 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.642007 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.646846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.646846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.646846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.646846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.661303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1329/2670] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:12:51.664317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.674761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f521de6-32c6-41c5-9a5e-c0b097832ad4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.680116 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.680718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.680997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.681382 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.681890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.683501 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.694847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1342/2671] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:51.699103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.709922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25076e5-693f-4bf2-823b-ab9215ea0a73 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.714647 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.715369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.715739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.716109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] PATH_INFO: `/v3/projects/95960a91408d4ce3b1dc8b0ff3dd3fd1/users/9e3547d3a0cb44128b7cba6fe8219a10/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.731697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95960a91408d4ce3b1dc8b0ff3dd3fd1, user_id=9e3547d3a0cb44128b7cba6fe8219a10, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.734485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac670-474c-4356-81e3-b478522c1412 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.769426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.771483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.791796 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bad5f-e190-499e-ae72-738d1ab571ce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:51.797397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:51.798022 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.798323 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.798634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] PATH_INFO: `/v3/projects/3ceb1ac7a84049f2b5264ca3734729ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.799742 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ceb1ac7a84049f2b5264ca3734729ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:51.802345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:51.843586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1330/2672] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:51 2022] PUT /identity/v3/projects/95960a91408d4ce3b1dc8b0ff3dd3fd1/users/9e3547d3a0cb44128b7cba6fe8219a10/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:51.854334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:51.854622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:51.854895 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:51.882949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] MFA Rules not processed for user `9e3547d3a0cb44128b7cba6fe8219a10`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:51.884202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:51.930194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ceb1ac7a84049f2b5264ca3734729ba', 'request_id': 'req-b41121bd-3585-4e98-bcd4-3692e255e5d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.933162 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:51.937868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ceb1ac7a84049f2b5264ca3734729ba', 'request_id': 'req-b41121bd-3585-4e98-bcd4-3692e255e5d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:51.957125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1331/2673] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:51.968922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:51.970290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:52.028346 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097acb92-271c-4caf-923d-6fa770e62364 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.042065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ceb1ac7a84049f2b5264ca3734729ba', 'request_id': 'req-b41121bd-3585-4e98-bcd4-3692e255e5d9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:52.053113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] The token cache is being invalidate because project 3ceb1ac7a84049f2b5264ca3734729ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:12:52.053361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:52.054000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b41121bd-3585-4e98-bcd4-3692e255e5d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:12:52.055551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1343/2674] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:12:51 2022] DELETE /identity/v3/projects/3ceb1ac7a84049f2b5264ca3734729ba => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:12:52.060174 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf0c8f-ee75-45f1-9064-9b4eb84d243c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.060613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf0c8f-ee75-45f1-9064-9b4eb84d243c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.060887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf0c8f-ee75-45f1-9064-9b4eb84d243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.079937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8adf0c8f-ee75-45f1-9064-9b4eb84d243c None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:52.084505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.085204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8adf0c8f-ee75-45f1-9064-9b4eb84d243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:52.085311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.085311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.085311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.085311 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:52.148506 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1344/2675] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:52.150400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.151914 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.157041 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.157326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.157597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.162231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1332/2676] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:12:52.177238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:52.178468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:52.187810 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1345/2677] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:12:52.190955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.203475 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2dfa-5e9a-4498-87f7-016b8d9b3c3e service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.208538 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.209171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.209448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.210300 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.210947 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:12:52.211248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.213189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.229520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1333/2678] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:12:52.230712 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.240360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b94b35-341c-456a-b080-32884b01d3a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.245826 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.246601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.246883 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.247328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.247983 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.250003 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.338347 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1346/2679] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:12:52.339611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.350736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c373-33c9-4657-afa1-ebe8d853f2e2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.394099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:52.410352 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.410962 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.411245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.411522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.412160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.414702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.511044 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1334/2680] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:52.515190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.526965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381bcf0-bcbf-4d93-b2de-b307af3b35b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.531746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.532376 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.532654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.532920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.533853 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.535612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.546557 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1347/2681] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:52.550830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.562144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd92f8-09fa-4c33-b19a-c50ffa7b3748 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.565696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.566387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.566684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.566959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.567594 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.569308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964d06b-56f1-43e7-985d-16df9c7d27db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.579912 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1335/2682] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:52.583979 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.594518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea8c94-f8a0-4ac0-acc0-955eb72fe8d9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.599227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.599925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.600996 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.601282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] PATH_INFO: `/v3/projects/1153d8617e4a4b0ca672726f6c0cec8b/users/e224e74f94ae459fbdcfb8e81498fce6/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.605156 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:52.616000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1153d8617e4a4b0ca672726f6c0cec8b, user_id=e224e74f94ae459fbdcfb8e81498fce6, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.618627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:52.710163 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1348/2683] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:52 2022] PUT /identity/v3/projects/1153d8617e4a4b0ca672726f6c0cec8b/users/e224e74f94ae459fbdcfb8e81498fce6/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:52.777133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.777440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.777699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.806347 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] MFA Rules not processed for user `e224e74f94ae459fbdcfb8e81498fce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:52.807557 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:52.874985 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1336/2684] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:52.876835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:52.937897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867fff1-a8d1-469e-8ea7-dd4450ac0bd5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:52.942594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:52.943202 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:52.943478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:52.943749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:52.947970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:52.947970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.026814 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1349/2685] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:52 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:53.031331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.042424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17f31d-f036-421d-85d7-9bfe72c489c9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.101433 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.102050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.102330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.102605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.103237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.105170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.179249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1337/2686] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:53 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:53.183271 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.194312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ceb89-5234-4688-89fe-a686ecd88fb9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.199176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.199811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.200095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.200365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.201000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.202616 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.213868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1350/2687] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:53.218237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.228122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480408-4577-47ec-8331-8f944acb860a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.232679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.233258 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.233531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.233797 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.234417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.236083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee429ca4-729e-4145-8ca6-b455bb0b8082 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.246233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1338/2688] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:53.250757 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.262887 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419db58d-f32f-4d63-9573-d995a4b72514 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.274279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.280138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.280138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.280138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] PATH_INFO: `/v3/projects/342556b896ef40d5a394fa8a14666a55/users/ba3687a09bec49cabe182039d3268c5e/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.295947 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=342556b896ef40d5a394fa8a14666a55, user_id=ba3687a09bec49cabe182039d3268c5e, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.298553 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.374037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1351/2689] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:53 2022] PUT /identity/v3/projects/342556b896ef40d5a394fa8a14666a55/users/ba3687a09bec49cabe182039d3268c5e/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:53.384796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.385088 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.385363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.416384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] MFA Rules not processed for user `ba3687a09bec49cabe182039d3268c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:53.417621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:53.437125 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:53.489315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1339/2690] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:53.494289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.554625 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7a8bc-fcb3-43f4-9885-897e4fe278d8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.559364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.560028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.560317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.560596 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.561216 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.563161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.687532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1352/2691] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:12:53 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:53.689307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.700415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629e725-ac5d-4320-b210-9771e488aff1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.759401 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.760043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.760328 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.760604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.761224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.763136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.917310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1340/2692] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:12:53 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:12:53.921301 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.933415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312979-7bc4-42fb-bd23-07394c22cbda admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.938606 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.939220 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.939498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.939808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.940478 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:53.942120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:53.953536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1353/2693] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:53.963617 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:53.974597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c76a6-edd3-4d84-a2e6-f98fe9494edc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:53.980188 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:53.980932 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:53.981220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:53.981495 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] PATH_INFO: `/v3/projects/a33d90d6d9524a8ebff0c221991f9318/users/91de1b04433544278803a0c02c268009/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:53.997697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33d90d6d9524a8ebff0c221991f9318, user_id=91de1b04433544278803a0c02c268009, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:54.000363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:54.111112 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1341/2694] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:53 2022] PUT /identity/v3/projects/a33d90d6d9524a8ebff0c221991f9318/users/91de1b04433544278803a0c02c268009/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:54.115259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:54.125259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3a07-6c80-4111-8c18-95ba39a1029e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:54.129977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:54.189036 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:54.189748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:54.190026 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:54.190292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:54.190924 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:54.192552 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdff55-d51b-4dcf-8eae-3e8f294e1cf5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:54.204024 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1354/2695] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:12:54.206995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:54.217849 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6933112-a27d-43da-af89-bf7d4633ddc3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:54.222469 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:54.223134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:54.223415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:54.223689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] PATH_INFO: `/v3/projects/a33d90d6d9524a8ebff0c221991f9318/users/91de1b04433544278803a0c02c268009/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:54.226245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33d90d6d9524a8ebff0c221991f9318, user_id=91de1b04433544278803a0c02c268009, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:54.228919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:54.238883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1342/2696] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:12:54 2022] PUT /identity/v3/projects/a33d90d6d9524a8ebff0c221991f9318/users/91de1b04433544278803a0c02c268009/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:12:54.250521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:54.250804 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:54.251074 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:54.279411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] MFA Rules not processed for user `91de1b04433544278803a0c02c268009`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:12:54.280637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:54.346352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1355/2697] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:12:54.359860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:54.361550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:54.420330 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfad04-ba97-4aa3-8a0f-7a85425eb55e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:54.423963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:54.424566 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:54.424838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:54.425104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:54.451662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:12:54.486374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:54.487911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:54.497656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1343/2698] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:56.630380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:56.640806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab309295-e88c-411a-9e22-550ed4c6bfd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:56.644352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:56.644952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:56.645227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:56.645493 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:56.703353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:56.704838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:56.713440 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1356/2699] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:12:58.298437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:12:58.299944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:12:58.311733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cd67c-69dc-43b7-8e78-24c1f12b0b82 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:12:58.315416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:12:58.316052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:12:58.316329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:12:58.316593 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:12:58.326389 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:12:58.327841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a248b-3a25-43e7-80e2-1744256344db service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:12:58.337396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1344/2700] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:12:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:03.057715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:03.088771 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:03.122266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b723-52fd-426f-9ca3-fdef55ac84be service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:03.126838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:03.127461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:03.127743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:03.128047 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] PATH_INFO: `/v3/users/e224e74f94ae459fbdcfb8e81498fce6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:03.129161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=e224e74f94ae459fbdcfb8e81498fce6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:03.131073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:03.383472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e224e74f94ae459fbdcfb8e81498fce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:03.384142 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e224e74f94ae459fbdcfb8e81498fce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:03.396101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e224e74f94ae459fbdcfb8e81498fce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:03.469681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e224e74f94ae459fbdcfb8e81498fce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:03.477496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e224e74f94ae459fbdcfb8e81498fce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:03.483515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1357/2701] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:03 2022] DELETE /identity/v3/users/e224e74f94ae459fbdcfb8e81498fce6 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:03.488010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13019366-6f66-4485-8954-320467989aae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:03.488514 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13019366-6f66-4485-8954-320467989aae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:03.488674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13019366-6f66-4485-8954-320467989aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:03.507417 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-13019366-6f66-4485-8954-320467989aae None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:03.508611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13019366-6f66-4485-8954-320467989aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:03.585632 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1345/2702] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:03.594050 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:03.595594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:03.658222 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc021aa-385b-4fe1-a227-4d0f96e6b7ce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:03.662004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:03.662641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:03.662944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:03.663185 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:03.672393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:03.673818 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:03.683294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1358/2703] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:04.135902 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13019366-6f66-4485-8954-320467989aae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:04.137362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13019366-6f66-4485-8954-320467989aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:04.149005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13019366-6f66-4485-8954-320467989aae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:04.155197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:04.156193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:04.156668 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:04.157099 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] PATH_INFO: `/v3/projects/1153d8617e4a4b0ca672726f6c0cec8b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:04.158780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1153d8617e4a4b0ca672726f6c0cec8b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:04.161069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:04.319716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1153d8617e4a4b0ca672726f6c0cec8b', 'request_id': 'req-c10a176f-4241-4fe1-a82c-d59ee50e209f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.327299 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1153d8617e4a4b0ca672726f6c0cec8b', 'request_id': 'req-c10a176f-4241-4fe1-a82c-d59ee50e209f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.418892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1153d8617e4a4b0ca672726f6c0cec8b', 'request_id': 'req-c10a176f-4241-4fe1-a82c-d59ee50e209f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.431577 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] The token cache is being invalidate because project 1153d8617e4a4b0ca672726f6c0cec8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:04.431899 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.432520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.433999 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1346/2704] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:04 2022] DELETE /identity/v3/projects/1153d8617e4a4b0ca672726f6c0cec8b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:04.437960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:04.447461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c6ce9-520d-4fdd-961c-c6bb01fa82dc service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:04.494059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:04.504902 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:04.505581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:04.505856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:04.506123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] PATH_INFO: `/v3/users/ba3687a09bec49cabe182039d3268c5e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:04.507213 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3687a09bec49cabe182039d3268c5e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:04.509227 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:04.764010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3687a09bec49cabe182039d3268c5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.764663 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3687a09bec49cabe182039d3268c5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.777062 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3687a09bec49cabe182039d3268c5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.853411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3687a09bec49cabe182039d3268c5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.863621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3687a09bec49cabe182039d3268c5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:04.871248 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1359/2705] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:04 2022] DELETE /identity/v3/users/ba3687a09bec49cabe182039d3268c5e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:05.385430 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:05.386922 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:05.399883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a176f-4241-4fe1-a82c-d59ee50e209f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:05.433574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:05.457428 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:05.458060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:05.458339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:05.458611 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] PATH_INFO: `/v3/projects/342556b896ef40d5a394fa8a14666a55` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:05.459724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=342556b896ef40d5a394fa8a14666a55)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:05.461644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:05.596348 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '342556b896ef40d5a394fa8a14666a55', 'request_id': 'req-2830976d-42e5-4b05-9635-3ad4ac170cad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.602810 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '342556b896ef40d5a394fa8a14666a55', 'request_id': 'req-2830976d-42e5-4b05-9635-3ad4ac170cad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.675906 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '342556b896ef40d5a394fa8a14666a55', 'request_id': 'req-2830976d-42e5-4b05-9635-3ad4ac170cad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.688035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] The token cache is being invalidate because project 342556b896ef40d5a394fa8a14666a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:05.688506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.689157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.690886 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1347/2706] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:05 2022] DELETE /identity/v3/projects/342556b896ef40d5a394fa8a14666a55 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:05.694773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:05.696258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:05.708714 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d5dcf-502f-43d5-8754-d4bf0ea6f8a5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:05.764296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:05.764918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:05.765198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:05.765476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] PATH_INFO: `/v3/users/91de1b04433544278803a0c02c268009` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:05.766607 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91de1b04433544278803a0c02c268009)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:05.768560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:05.953773 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91de1b04433544278803a0c02c268009'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.954442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91de1b04433544278803a0c02c268009'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:05.971321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91de1b04433544278803a0c02c268009'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.085636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91de1b04433544278803a0c02c268009'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.087175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:06.095294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fe47216-84e8-4ee2-ab71-50f30f284ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91de1b04433544278803a0c02c268009'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.102296 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1360/2707] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:05 2022] DELETE /identity/v3/users/91de1b04433544278803a0c02c268009 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:06.539982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:06.544805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:06.558569 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830976d-42e5-4b05-9635-3ad4ac170cad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:06.617919 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:06.618491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:06.620716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:06.620716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] PATH_INFO: `/v3/projects/a33d90d6d9524a8ebff0c221991f9318` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:06.620716 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33d90d6d9524a8ebff0c221991f9318)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:06.622045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:06.814171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33d90d6d9524a8ebff0c221991f9318', 'request_id': 'req-c4779c88-512b-4221-aec6-2bded818fafd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.821292 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33d90d6d9524a8ebff0c221991f9318', 'request_id': 'req-c4779c88-512b-4221-aec6-2bded818fafd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.932502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33d90d6d9524a8ebff0c221991f9318', 'request_id': 'req-c4779c88-512b-4221-aec6-2bded818fafd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.939318 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:06.961858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] The token cache is being invalidate because project a33d90d6d9524a8ebff0c221991f9318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:06.962145 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.962842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4779c88-512b-4221-aec6-2bded818fafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:06.964376 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1348/2708] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:06 2022] DELETE /identity/v3/projects/a33d90d6d9524a8ebff0c221991f9318 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:06.969151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:06.969662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:06.969767 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:06.990782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:06.992014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:07.055725 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1361/2709] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:07.063588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.063697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.064017 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.086341 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:07.087542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:07.095559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1349/2710] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:13:07.100556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.110445 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef907e92-3300-48f5-8805-58dbff6f0f55 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.115169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.115783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.116188 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.116454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.117092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:13:07.117390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.119059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.132132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1362/2711] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:13:07.137149 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.147350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd9829-4c41-4368-bf13-481eb533b8e6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.153042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.153658 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.153936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.154208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.154851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.156790 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.272135 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1350/2712] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:13:07 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:07.276431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.286895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1d8ba-f829-4656-b24c-f35be177dbc0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.342277 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.343008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.343317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.343591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.344292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.346217 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.453253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1363/2713] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:13:07 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:07.458027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.468642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f32e8e-add9-4a9b-bd3e-cde4e7867799 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.473269 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.473887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.474205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.474480 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.475123 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.476734 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.487526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1351/2714] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:07.495377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.506081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1791f1-9943-40fa-9b67-1e18877f6e94 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.510800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.511394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.511672 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.511977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.512613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.514190 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ef47-336f-464c-bf0b-3aec47d02631 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.525044 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1364/2715] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:07.529163 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.539585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265499f-f104-463e-bf1a-06e1ef835a0b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.544306 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.544969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.545242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.545508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] PATH_INFO: `/v3/projects/b84146bd4aa1425c843ad28653586d7a/users/172fb1dca7c24ffdba4d10b34ba45fe5/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.560134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84146bd4aa1425c843ad28653586d7a, user_id=172fb1dca7c24ffdba4d10b34ba45fe5, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.562842 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.652930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1352/2716] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:07 2022] PUT /identity/v3/projects/b84146bd4aa1425c843ad28653586d7a/users/172fb1dca7c24ffdba4d10b34ba45fe5/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:07.663045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579884b-3354-4689-88dc-f1b4ff273b0f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.663332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579884b-3354-4689-88dc-f1b4ff273b0f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.663634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579884b-3354-4689-88dc-f1b4ff273b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.682665 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f579884b-3354-4689-88dc-f1b4ff273b0f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:07.683878 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f579884b-3354-4689-88dc-f1b4ff273b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:07.732496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:07.744643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1365/2717] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:07.754835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:07.757171 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:07.816441 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ff15b-4e1d-4ae0-9483-beb36682271e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:07.821339 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:07.822524 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:07.823261 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:07.823544 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:07.833784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:07.835230 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:07.844526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1353/2718] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:09.894904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795a0c-4eb1-48bb-8d58-771006d81916 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:09.894904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795a0c-4eb1-48bb-8d58-771006d81916 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:09.894904 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795a0c-4eb1-48bb-8d58-771006d81916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:09.919040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-84795a0c-4eb1-48bb-8d58-771006d81916 None None] MFA Rules not processed for user `172fb1dca7c24ffdba4d10b34ba45fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:09.920383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84795a0c-4eb1-48bb-8d58-771006d81916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:09.986543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1366/2719] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:09.998329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:10.000242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:10.060936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97aa0c-ee03-435e-8830-012a5142df14 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:10.064770 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:10.065418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:10.065679 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:10.065949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:10.075418 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:10.077016 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:10.088250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1354/2720] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:11.932442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f219-fae5-4b54-a9fa-83381b50ef19 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:11.932956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f219-fae5-4b54-a9fa-83381b50ef19 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:11.932956 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f219-fae5-4b54-a9fa-83381b50ef19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:11.951985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c042f219-fae5-4b54-a9fa-83381b50ef19 None None] MFA Rules not processed for user `172fb1dca7c24ffdba4d10b34ba45fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:11.953169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c042f219-fae5-4b54-a9fa-83381b50ef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:11.960830 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1367/2721] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:12.737127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:12.737127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:12.744488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379bae-788b-4600-a3d4-bda59e1bf8d9 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:12.749294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:12.749673 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:12.750031 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:12.750268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] PATH_INFO: `/v3/users/9e3547d3a0cb44128b7cba6fe8219a10` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:12.751375 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3547d3a0cb44128b7cba6fe8219a10)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:12.753296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:12.989857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:13.011539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3547d3a0cb44128b7cba6fe8219a10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.012252 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3547d3a0cb44128b7cba6fe8219a10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.023520 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3547d3a0cb44128b7cba6fe8219a10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.031739 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3547d3a0cb44128b7cba6fe8219a10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.114933 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3547d3a0cb44128b7cba6fe8219a10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.121354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1355/2722] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:12 2022] DELETE /identity/v3/users/9e3547d3a0cb44128b7cba6fe8219a10 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:13.126790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:13.126790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:13.126790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:13.145547 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:13.147280 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:13.204840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1368/2723] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:13.215102 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:13.216513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:13.273313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaccfc9-e363-49a1-9318-423c92b7ca1a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:13.276844 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:13.279132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:13.279132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:13.279132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:13.287137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:13.288583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d795017-207e-406b-a286-08335fbc91fb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:13.298023 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1356/2724] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:13.803365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:13.806061 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:13.826448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463b466-476c-421d-a847-2c2c44a0bd0f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:13.831181 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:13.831869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:13.832154 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:13.832428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] PATH_INFO: `/v3/projects/95960a91408d4ce3b1dc8b0ff3dd3fd1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:13.833560 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95960a91408d4ce3b1dc8b0ff3dd3fd1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:13.835581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:13.986815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95960a91408d4ce3b1dc8b0ff3dd3fd1', 'request_id': 'req-57eed287-77c3-4a1e-a0d3-8baf8510839f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:13.993621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95960a91408d4ce3b1dc8b0ff3dd3fd1', 'request_id': 'req-57eed287-77c3-4a1e-a0d3-8baf8510839f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:14.093209 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95960a91408d4ce3b1dc8b0ff3dd3fd1', 'request_id': 'req-57eed287-77c3-4a1e-a0d3-8baf8510839f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:14.105103 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] The token cache is being invalidate because project 95960a91408d4ce3b1dc8b0ff3dd3fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:14.105392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:14.106046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57eed287-77c3-4a1e-a0d3-8baf8510839f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:14.107526 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1369/2725] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:13 2022] DELETE /identity/v3/projects/95960a91408d4ce3b1dc8b0ff3dd3fd1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:14.112362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.112854 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.112969 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.132600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:14.133824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:14.194025 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1357/2726] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:14.201855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.202341 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.202466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.209914 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:14.222939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:14.227861 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:14.232678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1370/2727] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:14.236930 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.247127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c167-96be-4e5e-a9c5-2f936885ea69 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.251753 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.252405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.252682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.252952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.253589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:13:14.253890 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.255463 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.267298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1358/2728] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:13:14.272151 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.281261 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6676ed5-b31a-4e3c-a015-2565499874c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.286089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.286669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.286959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.287748 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.288406 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.290262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.385373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1371/2729] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:14.389546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.399967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f93b3-0507-4fcc-baff-9c4d9e6b6633 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.437110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:14.455086 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.455689 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.456000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.456277 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.456892 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.458789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.580421 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1359/2730] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:14.585031 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.595977 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575b1d9-3b95-4e8a-b389-979be780a516 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.601706 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.602330 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.602676 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.602934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.603588 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.606424 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.615736 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1372/2731] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:14.619863 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.629972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6231ae-0b13-4dc4-a2f5-a69016287f87 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.635387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.639377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.639377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.639377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.639377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.639377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069171f-d772-4f32-93ae-f63ec3329cf3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.650399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1360/2732] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:14.654332 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:14.666383 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12251a4-2abc-43fd-8daf-6980173320a7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:14.671780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:14.672566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.672848 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.673120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] PATH_INFO: `/v3/projects/db802e5dde044399a2f4b58d7665b05e/users/cd849b9a73d0432fbc031a9a02b73790/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.687659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db802e5dde044399a2f4b58d7665b05e, user_id=cd849b9a73d0432fbc031a9a02b73790, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:14.690318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:14.779319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1373/2733] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:14 2022] PUT /identity/v3/projects/db802e5dde044399a2f4b58d7665b05e/users/cd849b9a73d0432fbc031a9a02b73790/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:13:14.845090 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:14.845377 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:14.845655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:14.872794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] MFA Rules not processed for user `cd849b9a73d0432fbc031a9a02b73790`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:14.873982 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:14.937363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1361/2734] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:14.941808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.002043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9e9b4-8a93-4e53-ae0d-d2020764ae31 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.007092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.007682 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.007985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.008252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.008869 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.010731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.140545 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1374/2735] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:15.144825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.157184 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f52b21-3477-4988-bf84-b4c109945365 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.217786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.218394 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.218676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.218949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.219562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.221512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.460259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1362/2736] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:13:15 2022] POST /identity/v3/users => generated 292 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:13:15.464305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.474537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e3799-21f2-4c0f-bd36-9fb5ed4b97e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.479483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.480203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.480477 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.480746 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.481380 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.482955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.493927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1375/2737] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:15.500063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.510198 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eba6e-3e1f-408a-b626-c9d9701e7f5a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.514954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.515704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.516037 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.516310 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.516980 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.518574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc5d24-6997-4dc0-ab0f-9fc29971eefb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.529144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1363/2738] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:15.533243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.543687 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e23ba-f9dc-4b45-b187-67a015143d9e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.548159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.548788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.549060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.549326 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] PATH_INFO: `/v3/projects/2d24e43bc8ef4b2fa83d524e7cf02648/users/d0ca1a6377f343e88f77675730ff39da/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.564108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d24e43bc8ef4b2fa83d524e7cf02648, user_id=d0ca1a6377f343e88f77675730ff39da, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.566730 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.677430 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1376/2739] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:15 2022] PUT /identity/v3/projects/2d24e43bc8ef4b2fa83d524e7cf02648/users/d0ca1a6377f343e88f77675730ff39da/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:15.688097 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.688455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.688802 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.717878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] MFA Rules not processed for user `d0ca1a6377f343e88f77675730ff39da`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:15.719170 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:15.732879 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:15.785320 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1364/2740] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:15.790355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.839520 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:15.852913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e09c3-a9a6-483e-9948-6108e3f5385c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:15.857837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:15.858426 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:15.858724 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:15.858981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:15.859597 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:15.861511 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:15.953032 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1377/2741] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:13:15 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:15.956878 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:15.968032 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5665a-064d-4094-b9a4-9bdd91ac3b1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.025151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.025765 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.026069 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.026342 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.026973 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.028945 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.147473 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1365/2742] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:13:15 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:16.152854 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:16.172075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed20b-1dc8-4085-a13e-71b3a566cb21 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.176958 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.177566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.177847 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.178120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.178794 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.180392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.195585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1378/2743] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:16.198764 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:16.209449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666251-00c2-4a85-88c4-b64c08c53d5e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.215254 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.218206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.218206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.218206 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] PATH_INFO: `/v3/projects/d854c446915244f0a35e3bd07b988240/users/864ca82b1fe140a7829fc2d96c4bfbfb/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.234122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d854c446915244f0a35e3bd07b988240, user_id=864ca82b1fe140a7829fc2d96c4bfbfb, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.236957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.314142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1366/2744] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:16 2022] PUT /identity/v3/projects/d854c446915244f0a35e3bd07b988240/users/864ca82b1fe140a7829fc2d96c4bfbfb/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:16.318828 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:16.330686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362f1e-7902-4ecc-b20b-1b6eb9e22911 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.391057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.391670 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.391986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.392259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.392957 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.394546 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50be7d-4aa5-4a1a-8b1f-5568bd9b8acb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.405372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1379/2745] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:16.409517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:16.420078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1f907-edc7-4943-9174-84208f37b5e0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.425026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.425500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.425781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.426052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] PATH_INFO: `/v3/projects/d854c446915244f0a35e3bd07b988240/users/864ca82b1fe140a7829fc2d96c4bfbfb/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.428073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d854c446915244f0a35e3bd07b988240, user_id=864ca82b1fe140a7829fc2d96c4bfbfb, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.430741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.441030 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1367/2746] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:16 2022] PUT /identity/v3/projects/d854c446915244f0a35e3bd07b988240/users/864ca82b1fe140a7829fc2d96c4bfbfb/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:16.452675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.452959 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.453662 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.481048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] MFA Rules not processed for user `864ca82b1fe140a7829fc2d96c4bfbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:16.482224 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:16.561381 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1380/2747] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:16.574053 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:16.575313 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:16.647343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51accf-7f7f-4fb6-9f50-36422ffb8e87 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:16.651754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:16.652225 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:16.652504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:16.652776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:16.675228 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:16.712707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:16.714142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:16.725959 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1368/2748] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:18.898393 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:18.900469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:18.913291 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738a8a-af93-471a-aa48-e0a2c6c1bec3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:18.917259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:18.917974 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:18.918368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:18.918707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:18.964678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:18.983622 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:18.985502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:18.996352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1381/2749] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:20.729493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:20.738586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509e8cc-d268-4eac-b7cd-b7132c3f0acd service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:20.742109 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:20.742690 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:20.742963 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:20.743229 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:20.753574 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:20.754966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08536c-57d5-4f6d-b8f5-2c1d630eaa83 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:20.764517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1369/2750] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:25.411496 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:25.476394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac6d6-3cca-42a7-a615-b5136f58ffa3 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:25.480937 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:25.481562 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:25.481837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:25.482107 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] PATH_INFO: `/v3/users/cd849b9a73d0432fbc031a9a02b73790` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:25.483206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd849b9a73d0432fbc031a9a02b73790)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:25.485167 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:25.682442 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd849b9a73d0432fbc031a9a02b73790'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:25.683096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd849b9a73d0432fbc031a9a02b73790'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:25.695121 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd849b9a73d0432fbc031a9a02b73790'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:25.765085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd849b9a73d0432fbc031a9a02b73790'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:25.773556 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd849b9a73d0432fbc031a9a02b73790'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:25.779790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1382/2751] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:25 2022] DELETE /identity/v3/users/cd849b9a73d0432fbc031a9a02b73790 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:25.784640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:25.786115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:25.786115 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:25.805039 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:25.806253 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:25.867609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1370/2752] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:13:25.877525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:25.879056 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:25.942115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cb38-0c87-4f18-8296-d78c32aa6646 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:25.945733 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:25.946340 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:25.946620 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:25.946898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:25.956315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:25.957740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:25.967198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1383/2753] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:13:26.406955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:26.409027 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:26.421150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7abc5-7d65-461e-aba1-a1102f4ec698 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:26.425804 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:26.426437 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:26.426719 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:26.427235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] PATH_INFO: `/v3/projects/db802e5dde044399a2f4b58d7665b05e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:26.428424 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db802e5dde044399a2f4b58d7665b05e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:26.430322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:26.560505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db802e5dde044399a2f4b58d7665b05e', 'request_id': 'req-bec26397-314f-4a34-b9ba-bf9fe6980be6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.566900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db802e5dde044399a2f4b58d7665b05e', 'request_id': 'req-bec26397-314f-4a34-b9ba-bf9fe6980be6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.668837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db802e5dde044399a2f4b58d7665b05e', 'request_id': 'req-bec26397-314f-4a34-b9ba-bf9fe6980be6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.680189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] The token cache is being invalidate because project db802e5dde044399a2f4b58d7665b05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:26.680477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.681128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.682622 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1371/2754] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:26 2022] DELETE /identity/v3/projects/db802e5dde044399a2f4b58d7665b05e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:26.686633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:26.693228 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:26.696966 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fe071-ea38-4f01-8a99-52158408a19a service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:26.750691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:26.751312 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:26.751602 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:26.752387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] PATH_INFO: `/v3/users/d0ca1a6377f343e88f77675730ff39da` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:26.753800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ca1a6377f343e88f77675730ff39da)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:26.756052 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:26.958070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ca1a6377f343e88f77675730ff39da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.958918 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ca1a6377f343e88f77675730ff39da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:26.974037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ca1a6377f343e88f77675730ff39da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.074756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ca1a6377f343e88f77675730ff39da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.082637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ca1a6377f343e88f77675730ff39da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.088960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1384/2755] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:26 2022] DELETE /identity/v3/users/d0ca1a6377f343e88f77675730ff39da => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:27.533989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:27.535638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:27.551515 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26397-314f-4a34-b9ba-bf9fe6980be6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:27.596169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:27.625768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:27.627137 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:27.627415 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:27.627709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] PATH_INFO: `/v3/projects/2d24e43bc8ef4b2fa83d524e7cf02648` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:27.628936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d24e43bc8ef4b2fa83d524e7cf02648)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:27.630921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:27.795605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d24e43bc8ef4b2fa83d524e7cf02648', 'request_id': 'req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.801794 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d24e43bc8ef4b2fa83d524e7cf02648', 'request_id': 'req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.901420 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d24e43bc8ef4b2fa83d524e7cf02648', 'request_id': 'req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.912997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] The token cache is being invalidate because project 2d24e43bc8ef4b2fa83d524e7cf02648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:27.913276 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.913898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:27.915374 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1372/2756] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:27 2022] DELETE /identity/v3/projects/2d24e43bc8ef4b2fa83d524e7cf02648 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:27.919638 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:27.921186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:27.933811 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2841a8-69ab-4420-8fd2-5d8f21c98e99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:27.997432 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:27.998060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:27.998364 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:27.998632 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] PATH_INFO: `/v3/users/864ca82b1fe140a7829fc2d96c4bfbfb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:27.999735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=864ca82b1fe140a7829fc2d96c4bfbfb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:28.001849 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:28.127521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:28.201889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864ca82b1fe140a7829fc2d96c4bfbfb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.202943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864ca82b1fe140a7829fc2d96c4bfbfb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.221234 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864ca82b1fe140a7829fc2d96c4bfbfb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.281206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864ca82b1fe140a7829fc2d96c4bfbfb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.288839 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-583b535c-d70a-4576-b7e9-f2a755801410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864ca82b1fe140a7829fc2d96c4bfbfb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.294987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1385/2757] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:27 2022] DELETE /identity/v3/users/864ca82b1fe140a7829fc2d96c4bfbfb => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:28.728724 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:28.732257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:28.744414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c9a95-0a6f-4461-8bd6-12f1e5f49feb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:28.804240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:28.804860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:28.805136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:28.805404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] PATH_INFO: `/v3/projects/d854c446915244f0a35e3bd07b988240` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:28.806540 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d854c446915244f0a35e3bd07b988240)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:28.808543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:28.928084 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd854c446915244f0a35e3bd07b988240', 'request_id': 'req-758fe779-4daf-4e82-8232-e4db3f14d126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.934823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd854c446915244f0a35e3bd07b988240', 'request_id': 'req-758fe779-4daf-4e82-8232-e4db3f14d126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:28.995993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd854c446915244f0a35e3bd07b988240', 'request_id': 'req-758fe779-4daf-4e82-8232-e4db3f14d126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:29.000073 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:29.007233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] The token cache is being invalidate because project d854c446915244f0a35e3bd07b988240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:29.007522 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:29.008210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-758fe779-4daf-4e82-8232-e4db3f14d126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:29.009779 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1373/2758] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:28 2022] DELETE /identity/v3/projects/d854c446915244f0a35e3bd07b988240 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:29.014718 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.015208 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.015319 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.036893 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:29.038120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:29.100285 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1386/2759] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:13:29.108625 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.109028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.109169 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.131621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:29.133020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:29.143429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1374/2760] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:29.146287 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.156570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5aa40e-f6be-42b9-a576-f3c91051bfcb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.161267 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.166465 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.180740 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1387/2761] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:13:29.181793 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.190774 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a4ee0-f06d-4751-9f75-aa4a2822bb00 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.195638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.202628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.202628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.202628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.202628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.202628 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.319870 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1375/2762] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:13:29 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:29.324138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.333855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff220ad0-66eb-4740-b01f-d4f1c45f6200 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.393373 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.393973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.394279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.394551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.395203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.397144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.585680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1388/2763] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:13:29 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:13:29.596568 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.600964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f8e3-ebd6-407b-ae59-8f9011183fe0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.608642 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.609243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.609519 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.609784 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.610419 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.612133 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.625052 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1376/2764] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:13:29.627161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.642399 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f710b8-c0df-49e0-9e6e-40d9397b67e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.643008 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.643608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.643927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.644205 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.644846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.646433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e24492-eaab-49f3-9371-c5d7c9b2799e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.658934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1389/2765] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:13:29.663340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.674110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1e0a1-9638-44ee-8b19-bee3e6b703dc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.678763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.679404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.679677 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.680001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] PATH_INFO: `/v3/projects/e894a0452b15498fbaa4ee42409cccea/users/247974fbb16348549ec3e04ef743b876/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.694126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e894a0452b15498fbaa4ee42409cccea, user_id=247974fbb16348549ec3e04ef743b876, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.697528 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:29.803869 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1377/2766] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:13:29 2022] PUT /identity/v3/projects/e894a0452b15498fbaa4ee42409cccea/users/247974fbb16348549ec3e04ef743b876/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:29.809037 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.809415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.809650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.829695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:29.838527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] MFA Rules not processed for user `247974fbb16348549ec3e04ef743b876`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:29.839758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:29.904788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1390/2767] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:29.916177 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:29.917984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:29.979676 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47347b76-e726-4427-9f86-f59b0e8d74fb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:29.983581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:29.984283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:29.984559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:29.984825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:29.994702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:29.996373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89701f-ec21-48ce-bc84-fd75279b5488 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:30.006002 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1378/2768] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:33.253637 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:33.319066 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cecd1-d44d-4765-91a4-0de3f257ae29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:33.323654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:33.324305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:33.328204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:33.328204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] PATH_INFO: `/v3/users/247974fbb16348549ec3e04ef743b876` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:33.328204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=247974fbb16348549ec3e04ef743b876)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:33.328204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:33.552101 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247974fbb16348549ec3e04ef743b876'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:33.552741 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247974fbb16348549ec3e04ef743b876'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:33.565124 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247974fbb16348549ec3e04ef743b876'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:33.645354 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247974fbb16348549ec3e04ef743b876'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:33.653086 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247974fbb16348549ec3e04ef743b876'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:33.658889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1391/2769] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:13:33 2022] DELETE /identity/v3/users/247974fbb16348549ec3e04ef743b876 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:13:33.663432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:33.668064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:33.668064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:33.683440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:33.684713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:33.742050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1379/2770] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:13:33.751713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:33.753129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:33.772981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:33.809293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c26d74-36a5-422c-9731-e7e016a4bb61 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:33.812882 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:33.813480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:33.813756 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:33.814028 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:33.825127 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:33.826584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58efdfc-1f4e-4709-839d-4d1c578fdf35 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:33.836292 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1392/2771] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:34.304303 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:34.305789 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:34.317887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2ae7-5f6a-4c13-afe5-56027540230a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:34.322491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:34.323117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:34.323404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:34.323686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] PATH_INFO: `/v3/projects/e894a0452b15498fbaa4ee42409cccea` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:34.325361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e894a0452b15498fbaa4ee42409cccea)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:34.327247 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:34.442583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:34.457714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e894a0452b15498fbaa4ee42409cccea', 'request_id': 'req-7801df38-961e-4ba7-8fa5-feb6adc4b498'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:34.463993 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e894a0452b15498fbaa4ee42409cccea', 'request_id': 'req-7801df38-961e-4ba7-8fa5-feb6adc4b498'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:34.557821 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e894a0452b15498fbaa4ee42409cccea', 'request_id': 'req-7801df38-961e-4ba7-8fa5-feb6adc4b498'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:34.558194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:34.558194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:34.558194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:34.565738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] The token cache is being invalidate because project e894a0452b15498fbaa4ee42409cccea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:13:34.566028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:34.567116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:13:34.568702 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1380/2772] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:13:34 2022] DELETE /identity/v3/projects/e894a0452b15498fbaa4ee42409cccea => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:13:34.572675 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] MFA Rules not processed for user `172fb1dca7c24ffdba4d10b34ba45fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:34.573926 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:34.641089 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1393/2773] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:13:37.561579 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:37.563138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:37.627273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801df38-961e-4ba7-8fa5-feb6adc4b498 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:37.631021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:37.631659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:37.631968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:37.632242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:37.634021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:37.736013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:37.737535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791d829-05a5-4427-8e21-fa7baa7d3d36 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:37.761384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1381/2774] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:39.084128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:13:39.094338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25254a-23dd-4d25-b064-2f92ca49a628 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:13:39.097863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:13:39.098463 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:39.098738 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:39.099004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:39.100745 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:39.114186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:13:39.115629 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7341d92-4dff-4973-927d-f31b5f25f3f8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:13:39.127454 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1394/2775] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:13:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:13:47.756638 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=80,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:49.121729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=49 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:13:52.637992 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:13:52.638542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:13:52.638542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:13:52.658240 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] MFA Rules not processed for user `08d81bcd5a4f43ce8b4839ee15cb163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:13:52.662305 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:13:52.731595 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1382/2776] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:14:00.312325 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:00.320236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:00.320236 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:00.335708 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:14:00.336962 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:00.408153 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1395/2777] 10.222.0.48 () {58 vars in 1009 bytes} [Thu Aug 11 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:14:00.547513 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:00.548952 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:00.624554 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7849805-ab33-47e8-aca0-7372e98d8fb3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:00.629539 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:00.630148 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:00.630541 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:00.630822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:00.639590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:00.641226 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14c0a1-3ba4-4d5a-8da0-ea5c430f9df5 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:00.649500 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1383/2778] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:14:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:14:10.399535 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=25 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:10.646243 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=57 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:14.516800 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:14.518358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:14.578585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b2fdb-3575-4114-ae4e-8f7468b1275b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:14.582313 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:14.583428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:14.583707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:14.584010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:14.585778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:14.652464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:14.653895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:14.663864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1396/2779] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:19.975222 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:19.975650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:19.975832 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:20.027792 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] MFA Rules not processed for user `8bf57a090d4f4ce5b950ccb1c1c584a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:14:20.029018 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:20.046028 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1384/2780] 10.222.0.48 () {58 vars in 967 bytes} [Thu Aug 11 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:14:20.058948 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:20.080758 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9250cc3-b5a3-4397-83d0-a03d2f788d28 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:20.085797 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0dea5e1e47b48b0b7623c6b7100279e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:20.086385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:20.086660 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:20.086934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:20.095203 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:20.096806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd455db-91d2-47ce-bcae-fd2823efa1bb service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:20.105294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1397/2781] 10.222.0.48 () {60 vars in 1384 bytes} [Thu Aug 11 11:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 11:14:24.236957 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:24.238400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:24.242630 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:24.243072 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:24.243186 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:24.261897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] MFA Rules not processed for user `2a9dbe3b7fdd482681b6ab94c26f4e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:14:24.263243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:24.270266 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1398/2782] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:14:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:14:24.306041 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada29c6-b1fb-4e3e-9f86-b8f1fed545db None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:24.311646 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:24.312132 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:24.312416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:24.312686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] PATH_INFO: `/v3/users/172fb1dca7c24ffdba4d10b34ba45fe5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:24.313960 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172fb1dca7c24ffdba4d10b34ba45fe5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:24.315889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:24.518340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172fb1dca7c24ffdba4d10b34ba45fe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:24.519805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172fb1dca7c24ffdba4d10b34ba45fe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:24.526245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172fb1dca7c24ffdba4d10b34ba45fe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:24.539268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172fb1dca7c24ffdba4d10b34ba45fe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:24.595537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:24.606571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802ed60-7796-4716-b82a-ca03a210b322 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:24.610136 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:24.610729 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:24.611010 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:24.611274 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:24.612993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:24.617049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172fb1dca7c24ffdba4d10b34ba45fe5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:24.622631 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:24.623550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1385/2783] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:14:24 2022] DELETE /identity/v3/users/172fb1dca7c24ffdba4d10b34ba45fe5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:14:24.626735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:24.632923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1399/2784] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 11:14:25.032160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:25.034147 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:25.096103 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:25.117634 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ca9a3-c679-4215-8e03-4483d8f6d922 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:25.123383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:25.123862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:25.124151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:25.124454 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] PATH_INFO: `/v3/projects/b84146bd4aa1425c843ad28653586d7a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:25.125600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84146bd4aa1425c843ad28653586d7a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:25.127470 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:25.246958 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84146bd4aa1425c843ad28653586d7a', 'request_id': 'req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:25.253061 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84146bd4aa1425c843ad28653586d7a', 'request_id': 'req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:25.350122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84146bd4aa1425c843ad28653586d7a', 'request_id': 'req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:25.361110 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] The token cache is being invalidate because project b84146bd4aa1425c843ad28653586d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:14:25.361400 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:25.362074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:25.363581 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1386/2785] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:14:25 2022] DELETE /identity/v3/projects/b84146bd4aa1425c843ad28653586d7a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:14:25.510589 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:25.513513 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:25.598123 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b5ec6-38ff-4b7f-a423-811b2eb8fd27 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:25.612004 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:25.663253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:25.663889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:25.664169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:25.664436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:25.666158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:25.677045 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:25.678595 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:25.688659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1400/2786] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:29.422560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:29.424101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:29.487656 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32b02a-1f27-4ce2-b273-9df9951bd4a0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:29.491355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:29.491994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:29.492274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:29.492546 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:29.494255 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:29.504355 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:29.505936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:29.514391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1387/2787] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:30.291832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:30.293906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:30.370764 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3054567-b689-4190-b37f-6e8f32dd93ac service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:30.376230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:30.376856 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:30.380328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:30.380328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:30.380328 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:30.391443 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:30.393046 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:30.403715 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1401/2788] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:39.510498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:40.399075 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=59 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:45.068324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:45.078176 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2085dd-13da-41b4-88d4-8cea3250036e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:45.081966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:45.082597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:45.082858 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:45.083128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:45.084904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:45.146245 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:45.147825 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:45.157824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1388/2789] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:46.524177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:46.526317 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:46.545754 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a678ce6-675e-422e-be5e-074871fe252e service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:46.559293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:46.560669 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:46.561097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:46.561505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:46.565814 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:46.580939 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:46.582795 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:46.610634 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1402/2790] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:48.076984 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:48.090263 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cd638-129c-446f-8896-c7d3934b19d8 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:48.095948 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd622a3b3cc2a4e469033cb2d5c66d2ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:48.096550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:48.096827 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:48.097095 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:48.099648 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:48.162972 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:48.165128 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:48.174233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1389/2791] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:51.627509 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:51.637723 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24074a-f067-4721-a7bf-be82b20f844f service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:51.641544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:51.642148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:51.642431 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:51.642709 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:51.644467 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:51.710385 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:51.711981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:51.723064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1403/2792] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:14:58.170186 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=56 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:14:59.553733 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:14:59.555268 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:14:59.569929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66acddfa-99fb-4560-9f6e-a64d8719b731 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:14:59.574534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:14:59.575194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:14:59.575771 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:14:59.576071 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] PATH_INFO: `/v3/users/7b2f7b56f6d94908bf00b3f21954776c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:14:59.577203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2f7b56f6d94908bf00b3f21954776c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:14:59.579201 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:14:59.778710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2f7b56f6d94908bf00b3f21954776c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:59.779362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2f7b56f6d94908bf00b3f21954776c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:59.786460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2f7b56f6d94908bf00b3f21954776c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:59.799488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2f7b56f6d94908bf00b3f21954776c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:59.952264 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2f7b56f6d94908bf00b3f21954776c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:14:59.959699 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1390/2793] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:14:59 2022] DELETE /identity/v3/users/7b2f7b56f6d94908bf00b3f21954776c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:00.258420 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:00.268537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07a1de-7df0-45c0-841a-f60c4939964b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:00.326790 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:00.327342 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:00.327617 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:00.327922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] PATH_INFO: `/v3/projects/cea40881648b4df393fc35c07d8f68dd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:00.329614 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea40881648b4df393fc35c07d8f68dd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:00.331530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:00.467798 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea40881648b4df393fc35c07d8f68dd', 'request_id': 'req-aa736e85-1fe5-46b9-8d2a-0b70892a5823'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.474251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea40881648b4df393fc35c07d8f68dd', 'request_id': 'req-aa736e85-1fe5-46b9-8d2a-0b70892a5823'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.565276 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea40881648b4df393fc35c07d8f68dd', 'request_id': 'req-aa736e85-1fe5-46b9-8d2a-0b70892a5823'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.576521 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] The token cache is being invalidate because project cea40881648b4df393fc35c07d8f68dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:00.576858 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.577439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.579007 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1404/2794] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:00 2022] DELETE /identity/v3/projects/cea40881648b4df393fc35c07d8f68dd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:00.582964 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:00.584436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:00.599547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7829-6c81-421d-96be-554c3ba8895c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:00.657008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:00.657624 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:00.657897 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:00.658189 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] PATH_INFO: `/v3/users/6fa2f8b2921f487b903d8ccff589950a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:00.659410 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa2f8b2921f487b903d8ccff589950a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:00.661331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:00.876801 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa2f8b2921f487b903d8ccff589950a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.877404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa2f8b2921f487b903d8ccff589950a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.884478 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa2f8b2921f487b903d8ccff589950a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.896353 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa2f8b2921f487b903d8ccff589950a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.898141 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:00.953966 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa2f8b2921f487b903d8ccff589950a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:00.960297 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1391/2795] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:00 2022] DELETE /identity/v3/users/6fa2f8b2921f487b903d8ccff589950a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:01.242038 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:01.243459 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:01.256437 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa736e85-1fe5-46b9-8d2a-0b70892a5823 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:01.265131 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:01.312821 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:01.313456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:01.313749 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:01.314230 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] PATH_INFO: `/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:01.315838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa5a06a02f7496cb5e7fcdb4efc4267)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:01.317787 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:01.497051 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa5a06a02f7496cb5e7fcdb4efc4267', 'request_id': 'req-c568fa31-799d-4447-96d4-4eab9afcd6f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:01.503229 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa5a06a02f7496cb5e7fcdb4efc4267', 'request_id': 'req-c568fa31-799d-4447-96d4-4eab9afcd6f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:01.640532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa5a06a02f7496cb5e7fcdb4efc4267', 'request_id': 'req-c568fa31-799d-4447-96d4-4eab9afcd6f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:01.651397 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] The token cache is being invalidate because project 1aa5a06a02f7496cb5e7fcdb4efc4267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:01.651689 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:01.652351 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:01.653837 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1405/2796] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:01 2022] DELETE /identity/v3/projects/1aa5a06a02f7496cb5e7fcdb4efc4267 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:11.169761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:11.648375 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:12.390083 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:12.391489 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:12.448196 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7738-d385-4b60-8b97-1f45834bfca8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:12.495542 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:12.496168 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:12.496448 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:12.496717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:12.498429 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:12.558754 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:12.560134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:12.569488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1392/2797] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:16.212658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:16.221568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:16.228334 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568fa31-799d-4447-96d4-4eab9afcd6f0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:16.232895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:16.233561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:16.233840 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:16.234110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] PATH_INFO: `/v3/users/08d81bcd5a4f43ce8b4839ee15cb163b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:16.235245 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d81bcd5a4f43ce8b4839ee15cb163b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:16.237204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:16.459226 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d81bcd5a4f43ce8b4839ee15cb163b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:16.459833 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d81bcd5a4f43ce8b4839ee15cb163b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:16.471358 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d81bcd5a4f43ce8b4839ee15cb163b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:16.565233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d81bcd5a4f43ce8b4839ee15cb163b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:16.573278 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-98980ee0-4c55-44ff-84a4-22a9daab5c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d81bcd5a4f43ce8b4839ee15cb163b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:16.579411 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1406/2798] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:16 2022] DELETE /identity/v3/users/08d81bcd5a4f43ce8b4839ee15cb163b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:16.902674 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:16.914014 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbeb7d-dd46-41bf-a6f5-a6873b6e1d4c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:16.981893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:16.982525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:16.982805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:16.983079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] PATH_INFO: `/v3/projects/c56bd2a662e34a6395117027b5c28c5b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:16.984231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56bd2a662e34a6395117027b5c28c5b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:16.986121 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:16.995600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:17.136840 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56bd2a662e34a6395117027b5c28c5b', 'request_id': 'req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:17.142908 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56bd2a662e34a6395117027b5c28c5b', 'request_id': 'req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:17.232504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56bd2a662e34a6395117027b5c28c5b', 'request_id': 'req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:17.243155 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] The token cache is being invalidate because project c56bd2a662e34a6395117027b5c28c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:17.243442 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:17.247072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b4907b3-7ae4-406a-8b8b-fd8aa7d64e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:17.247072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1393/2799] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:16 2022] DELETE /identity/v3/projects/c56bd2a662e34a6395117027b5c28c5b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:17.250457 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.250942 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.251049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.269963 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:17.271169 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:17.328211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1407/2800] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:17.336759 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.337050 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.337329 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.357425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:17.358612 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:17.366361 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1394/2801] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:17.370585 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.380175 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8ff90-3aaf-4438-b16f-2d8ea9c65a1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.385232 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.385820 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.386097 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.386365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.387059 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:15:17.387362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.388978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.400897 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1408/2802] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:15:17.405078 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.414835 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3cce-02c8-4b8e-ace0-964431a7c1d8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.419491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.425655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.425655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.425655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.425655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.425655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.515334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1395/2803] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:17 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:17.519369 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.529985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776bdf0-e7ca-4ae9-a603-c620dfd6088d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.585921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.586527 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.586802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.587073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.587698 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.589639 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.597517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:17.707349 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1409/2804] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:17 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:17.712116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.722185 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1c83-61a1-48b4-a16c-99a6a7ca8562 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.726843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.727427 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.727704 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.728020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.728937 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.730662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.740476 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1396/2805] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:17.744612 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.754609 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d15ca-0d52-4957-908f-831b01a9248f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.759903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.760484 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.760761 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.761030 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.761659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.763233 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb89edc-07f8-48d7-a7a4-bb943ce3acd0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.774806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1410/2806] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:17.778720 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:17.788506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742f186-7bb9-4da7-99ac-3065ee08fdad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:17.793049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:17.793696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.793975 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.794249 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] PATH_INFO: `/v3/projects/30d5f410425244c2b41d58a621fa6b4f/users/e9fcb4a457f749fba031eb6121836cd5/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.809280 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=30d5f410425244c2b41d58a621fa6b4f, user_id=e9fcb4a457f749fba031eb6121836cd5, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:17.811921 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:17.895573 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1397/2807] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:17 2022] PUT /identity/v3/projects/30d5f410425244c2b41d58a621fa6b4f/users/e9fcb4a457f749fba031eb6121836cd5/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:17.906911 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:17.907264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:17.907601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:17.935448 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] MFA Rules not processed for user `e9fcb4a457f749fba031eb6121836cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:17.936667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:17.998469 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1411/2808] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:18.003013 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.063618 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d13bc-0852-4f55-ae81-db9f1e0ad6ec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.068126 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.069283 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.069560 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.069833 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.070458 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.072379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.167547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1398/2809] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:18 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:18.171398 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.182594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23db8e6-1700-4867-b7e4-c68ea7fe6177 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.238466 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.239114 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.239391 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.239659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.240353 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.242889 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.342768 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1412/2810] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:18 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:18.346659 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.357861 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2954f02-a007-4cb8-93dc-a32a144e9f95 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.362502 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.363130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.363407 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.363678 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.364373 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.370620 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.377005 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1399/2811] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:18.381122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.391093 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5ad9f-b16c-4a96-9fbb-6bde1467d38c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.395783 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.396429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.398780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.398780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] PATH_INFO: `/v3/projects/3d9bccc753944abaa276881dc0469722/users/920ec0702efa41fa8719456f070f303f/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.415641 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9bccc753944abaa276881dc0469722, user_id=920ec0702efa41fa8719456f070f303f, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.418367 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.485731 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1413/2812] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:18 2022] PUT /identity/v3/projects/3d9bccc753944abaa276881dc0469722/users/920ec0702efa41fa8719456f070f303f/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:18.489943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.500640 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff5413-061e-4cfc-828b-673171f37a79 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.504159 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:18.555803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.556396 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.556671 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.556939 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.557571 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.559129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7ca44-fd56-4da4-8f1a-5ad969ed6915 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.568517 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1400/2813] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:18.572777 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.583250 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4061224-aea2-4faf-ab7d-9e16ee0675f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.588488 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.589139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.589422 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.589695 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] PATH_INFO: `/v3/projects/3d9bccc753944abaa276881dc0469722/users/920ec0702efa41fa8719456f070f303f/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.591667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9bccc753944abaa276881dc0469722, user_id=920ec0702efa41fa8719456f070f303f, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.594372 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.610290 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1414/2814] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:18 2022] PUT /identity/v3/projects/3d9bccc753944abaa276881dc0469722/users/920ec0702efa41fa8719456f070f303f/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:18.620609 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.620898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.621179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.650106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] MFA Rules not processed for user `920ec0702efa41fa8719456f070f303f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:18.651308 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:18.718049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1401/2815] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:18.729885 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.731561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:18.754238 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:18.789293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e0c85-cb9a-4310-8b38-c0bb3b4e4c01 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.792913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.793601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.794082 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.794551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.804159 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.805657 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.815048 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1415/2816] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:18 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:18.861009 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:18.862405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:18.874057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e907ee-10c7-4970-bd6d-44e0b823d5f2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:18.877589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:18.878166 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:18.878439 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:18.878706 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:18.934380 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:18.935823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13576e73-28d1-4ac8-ba6b-0258d3950003 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:18.945000 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1402/2817] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:23.662817 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:23.724691 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c2083-01ab-4e29-97f6-80dc3bca1489 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:23.729311 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:23.730231 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:23.730949 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:23.731370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] PATH_INFO: `/v3/users/e9fcb4a457f749fba031eb6121836cd5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:23.733043 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fcb4a457f749fba031eb6121836cd5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:23.736108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:23.952497 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fcb4a457f749fba031eb6121836cd5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:23.954054 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fcb4a457f749fba031eb6121836cd5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:23.965633 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fcb4a457f749fba031eb6121836cd5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.040256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fcb4a457f749fba031eb6121836cd5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.048098 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fcb4a457f749fba031eb6121836cd5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.054023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1416/2818] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:23 2022] DELETE /identity/v3/users/e9fcb4a457f749fba031eb6121836cd5 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:24.058707 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:24.059237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:24.059237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:24.079091 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:24.080327 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:24.100203 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:24.141438 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1403/2819] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:24.151580 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:24.153001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:24.198515 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:24.209788 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc6bf4-9e17-403d-b934-48541d9838d0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:24.213932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:24.214525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:24.216481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:24.216481 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:24.223825 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:24.225253 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:24.235100 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1417/2820] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:24.661136 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:24.662559 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:24.674259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11a5a8-2088-4e56-9732-f0c757c06346 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:24.678882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:24.679499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:24.679788 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:24.680161 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] PATH_INFO: `/v3/projects/30d5f410425244c2b41d58a621fa6b4f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:24.681796 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30d5f410425244c2b41d58a621fa6b4f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:24.683684 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:24.795100 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30d5f410425244c2b41d58a621fa6b4f', 'request_id': 'req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.801146 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30d5f410425244c2b41d58a621fa6b4f', 'request_id': 'req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.882116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30d5f410425244c2b41d58a621fa6b4f', 'request_id': 'req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.893343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] The token cache is being invalidate because project 30d5f410425244c2b41d58a621fa6b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:24.893629 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.894257 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:24.895726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1404/2821] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:24 2022] DELETE /identity/v3/projects/30d5f410425244c2b41d58a621fa6b4f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:24.899973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:24.910000 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2cdfb-5b64-4d71-8b4e-a33e7ea845e2 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:24.964968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:24.965591 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:24.965923 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:24.966198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] PATH_INFO: `/v3/users/920ec0702efa41fa8719456f070f303f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:24.967303 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920ec0702efa41fa8719456f070f303f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:24.969251 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:25.204539 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920ec0702efa41fa8719456f070f303f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.205206 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920ec0702efa41fa8719456f070f303f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.217387 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920ec0702efa41fa8719456f070f303f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.288126 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920ec0702efa41fa8719456f070f303f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.295294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ed30e9b9-2730-4b61-acde-606829e58e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920ec0702efa41fa8719456f070f303f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.301080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1418/2822] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:24 2022] DELETE /identity/v3/users/920ec0702efa41fa8719456f070f303f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:25.763904 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:25.765366 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:25.778205 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5cc3b-d489-46ef-9cd5-1425031e1bc5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:25.835563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:25.836233 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:25.836518 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:25.836781 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] PATH_INFO: `/v3/projects/3d9bccc753944abaa276881dc0469722` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:25.837898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9bccc753944abaa276881dc0469722)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:25.839813 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:25.940550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9bccc753944abaa276881dc0469722', 'request_id': 'req-f6767ce9-df04-42c0-9154-6486f4c903d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:25.942384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:25.946900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9bccc753944abaa276881dc0469722', 'request_id': 'req-f6767ce9-df04-42c0-9154-6486f4c903d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:26.020379 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9bccc753944abaa276881dc0469722', 'request_id': 'req-f6767ce9-df04-42c0-9154-6486f4c903d2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:26.031363 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] The token cache is being invalidate because project 3d9bccc753944abaa276881dc0469722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:26.031650 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:26.032343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f6767ce9-df04-42c0-9154-6486f4c903d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:26.033845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1405/2823] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:25 2022] DELETE /identity/v3/projects/3d9bccc753944abaa276881dc0469722 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:26.038868 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.039368 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.039489 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.059256 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:26.060494 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:26.129139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1419/2824] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:26.137660 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.137940 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.138210 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.157516 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:26.158949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:26.166756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1406/2825] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:26.170934 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.176922 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:26.180568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc26a6-bef9-4c4b-9696-6accaf9917c6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.185686 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.186283 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.186561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.186834 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.187476 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:15:26.187801 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.189439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.201414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1420/2826] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:15:26.205782 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.215208 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035324b4-806f-4ffe-8d04-b9fc0d6d9e5e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.219713 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.230468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.230468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.230468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.230468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.230468 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.330834 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1407/2827] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:15:26.334899 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.344608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeefef-a9bd-4f69-a4d6-f2a92b1634ec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.404371 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.404981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.405258 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.405529 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.406148 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.408085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.489104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1421/2828] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:15:26.493237 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.504142 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7aea-b304-43ce-9b66-7a0159b00cc2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.508652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.509234 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.509508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.509776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.510447 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.512055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.521911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1408/2829] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:26.536329 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.546679 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5aedd-8552-4520-a412-e7b330806489 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.551204 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.551815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.552092 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.552359 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.552986 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.554551 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64380044-69d3-43b7-a0a2-838defc6dd6d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.565464 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1422/2830] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:26.569525 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.579885 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc3e7f-679d-4a90-a272-fbf187db6809 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.584589 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.585199 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.585477 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.585750 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] PATH_INFO: `/v3/projects/af933a3cbc874c3498f56c5b3bff7e8c/users/f35b3ca43d944d4cbf54d40841ba4302/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.601113 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af933a3cbc874c3498f56c5b3bff7e8c, user_id=f35b3ca43d944d4cbf54d40841ba4302, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.604302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.684220 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1409/2831] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:26 2022] PUT /identity/v3/projects/af933a3cbc874c3498f56c5b3bff7e8c/users/f35b3ca43d944d4cbf54d40841ba4302/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:15:26.695681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.695997 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.696255 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.724244 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] MFA Rules not processed for user `f35b3ca43d944d4cbf54d40841ba4302`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:26.725456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:26.789394 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1423/2832] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:26.795157 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.855197 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b44b4-586f-43f7-9a3a-74c5f05f784b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:26.859955 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:26.860563 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:26.860823 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:26.861093 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:26.861709 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:26.863576 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:26.958653 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1410/2833] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:26.962608 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:26.973674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06597f3e-60b1-422f-a95c-b26c98212057 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.029207 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.029807 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.030129 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.030402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.031009 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.033017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.040806 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:27.099543 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1424/2834] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:26 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:27.103586 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:27.114698 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e7a7c-e0fc-476d-9cda-7dceccc2f152 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.119399 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.120057 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.120369 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.120621 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.121267 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.122839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.134467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1411/2835] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 11:15:27.138449 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:27.148279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46912f36-eec0-4b85-8991-7994eef8e473 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.153395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.154029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.154302 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.154568 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] PATH_INFO: `/v3/projects/8ff9baef274a44fcbfb302ae669f241c/users/d4a09b558b1347e6bd87bb0c909bc96b/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.169294 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff9baef274a44fcbfb302ae669f241c, user_id=d4a09b558b1347e6bd87bb0c909bc96b, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.171921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.236993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1425/2836] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:27 2022] PUT /identity/v3/projects/8ff9baef274a44fcbfb302ae669f241c/users/d4a09b558b1347e6bd87bb0c909bc96b/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:15:27.240830 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:27.250596 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d19dd-b601-445b-afb2-b1f3b24c8e05 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.296506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:27.305860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.306450 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.306727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.306997 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.307623 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.309216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a3464-ba57-4c39-be1b-c98cd19ce37b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.318565 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1412/2837] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:27.323155 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:27.333441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c75a2-a5af-4a1e-b38c-a7398e5be835 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.338115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.339370 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.339643 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.340014 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] PATH_INFO: `/v3/projects/8ff9baef274a44fcbfb302ae669f241c/users/d4a09b558b1347e6bd87bb0c909bc96b/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.341955 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff9baef274a44fcbfb302ae669f241c, user_id=d4a09b558b1347e6bd87bb0c909bc96b, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.344647 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.354737 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1426/2838] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:27 2022] PUT /identity/v3/projects/8ff9baef274a44fcbfb302ae669f241c/users/d4a09b558b1347e6bd87bb0c909bc96b/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:27.365026 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.365256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.365534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.392871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] MFA Rules not processed for user `d4a09b558b1347e6bd87bb0c909bc96b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:27.394064 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:27.460460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1413/2839] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:27.471626 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:27.473361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:27.529954 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014a640-80f4-4dd9-a09a-37ebcbfaacdb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:27.533648 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:27.534243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:27.534517 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:27.534781 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:27.543518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:27.544998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00701a19-32c9-4403-904a-58163a80da65 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:27.554087 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1427/2840] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:28.761851 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:28.825224 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd7b1-c438-4c59-8175-6194b5450d22 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:28.829683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:28.830298 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:28.830572 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:28.830860 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] PATH_INFO: `/v3/users/f35b3ca43d944d4cbf54d40841ba4302` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:28.832049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35b3ca43d944d4cbf54d40841ba4302)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:28.833934 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:29.091696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35b3ca43d944d4cbf54d40841ba4302'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:29.092364 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35b3ca43d944d4cbf54d40841ba4302'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:29.099350 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35b3ca43d944d4cbf54d40841ba4302'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:29.112530 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35b3ca43d944d4cbf54d40841ba4302'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:29.114762 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:29.260954 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35b3ca43d944d4cbf54d40841ba4302'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:29.267049 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1414/2841] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:28 2022] DELETE /identity/v3/users/f35b3ca43d944d4cbf54d40841ba4302 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:29.271502 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:29.272034 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:29.272141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:29.290694 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:29.292534 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:29.348993 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1428/2842] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:29.359652 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:29.361131 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:29.416129 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b1d6b-9a49-458c-9cee-a45eb91b7ee3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:29.420290 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:29.420875 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:29.421151 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:29.421414 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:29.429928 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:29.431322 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:29.440436 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1415/2843] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:29.855970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:29.857654 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:29.869739 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740a28d-d825-4220-bb05-5ec579f2fdfd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:29.874305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:29.874968 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:29.875195 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:29.875571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] PATH_INFO: `/v3/projects/af933a3cbc874c3498f56c5b3bff7e8c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:29.876681 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af933a3cbc874c3498f56c5b3bff7e8c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:29.878571 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:30.007896 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af933a3cbc874c3498f56c5b3bff7e8c', 'request_id': 'req-6135ea71-55d7-4259-8a5c-a35896be9d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.008973 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:30.014480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af933a3cbc874c3498f56c5b3bff7e8c', 'request_id': 'req-6135ea71-55d7-4259-8a5c-a35896be9d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.106214 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af933a3cbc874c3498f56c5b3bff7e8c', 'request_id': 'req-6135ea71-55d7-4259-8a5c-a35896be9d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.117905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] The token cache is being invalidate because project af933a3cbc874c3498f56c5b3bff7e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:30.118909 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.119115 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.120653 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1429/2844] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:29 2022] DELETE /identity/v3/projects/af933a3cbc874c3498f56c5b3bff7e8c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:30.124826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:30.134714 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c997fe-3077-4440-b525-98fc7f5de94a service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:30.192893 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:30.193534 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:30.193798 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:30.194076 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] PATH_INFO: `/v3/users/d4a09b558b1347e6bd87bb0c909bc96b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:30.195182 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a09b558b1347e6bd87bb0c909bc96b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:30.197139 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:30.423351 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a09b558b1347e6bd87bb0c909bc96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.424001 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a09b558b1347e6bd87bb0c909bc96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.430738 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a09b558b1347e6bd87bb0c909bc96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.443006 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a09b558b1347e6bd87bb0c909bc96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.506498 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-745cc2a4-3648-4a96-aa95-f9780aae220c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a09b558b1347e6bd87bb0c909bc96b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:30.512944 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1416/2845] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:30 2022] DELETE /identity/v3/users/d4a09b558b1347e6bd87bb0c909bc96b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:30.953377 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:30.954873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:30.969627 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ea71-55d7-4259-8a5c-a35896be9d9e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.029645 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.030262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.030537 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.030802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] PATH_INFO: `/v3/projects/8ff9baef274a44fcbfb302ae669f241c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.031925 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff9baef274a44fcbfb302ae669f241c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.033802 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.210621 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff9baef274a44fcbfb302ae669f241c', 'request_id': 'req-3f25a2d8-d519-4492-960c-8cb46c83ac14'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:31.217636 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff9baef274a44fcbfb302ae669f241c', 'request_id': 'req-3f25a2d8-d519-4492-960c-8cb46c83ac14'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:31.313318 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff9baef274a44fcbfb302ae669f241c', 'request_id': 'req-3f25a2d8-d519-4492-960c-8cb46c83ac14'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:31.324095 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] The token cache is being invalidate because project 8ff9baef274a44fcbfb302ae669f241c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:31.329864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:31.329864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3f25a2d8-d519-4492-960c-8cb46c83ac14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:31.329864 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1430/2846] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:30 2022] DELETE /identity/v3/projects/8ff9baef274a44fcbfb302ae669f241c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:31.331526 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.332019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.332119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.351884 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:31.353079 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:31.362256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:31.411987 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1417/2847] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:31.421352 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.421635 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.421910 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.443408 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:31.444835 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:31.453932 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1431/2848] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:15:31.458390 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.468193 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88764d6-fd6c-4dcd-8430-50dd5edb463a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.472780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.473368 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.473655 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.473917 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.478737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:15:31.478737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.478737 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.488725 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1418/2849] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:15:31.493035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.502719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c124553-2733-401e-956f-fe54ce633e23 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.507279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.507895 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.508176 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.508444 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.509065 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.510944 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.577613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1432/2850] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:31 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:31.581887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.592158 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dab59-a1a5-4beb-b67c-47c0004cd96b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.649174 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.649766 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.650040 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.650307 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.650929 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.652841 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.730443 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1419/2851] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:31 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 11:15:31.735419 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.748365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78362503-67ba-46a0-b94e-6553a0492615 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.752945 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.753541 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.753815 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.754083 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.754722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.756345 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.766906 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1433/2852] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:31.770943 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.781343 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8e8f8-89f2-45b9-be25-52753082c128 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.786334 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.786925 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.787200 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.787466 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.788242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.789783 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8672198-80f3-45ee-9070-8cfadbcfa979 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.801667 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1420/2853] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:31.808512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:31.818386 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff148270-9bf9-48c3-ad06-3e9c38a69171 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:31.823033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:31.823667 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.823978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.824252 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] PATH_INFO: `/v3/projects/a68bc982b9fe4557bde0474ebc4eb513/users/02c9c17918f9425baf5cd0f1fc1150d1/roles/933643980c98414dad5713dc8820df36` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:31.839141 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68bc982b9fe4557bde0474ebc4eb513, user_id=02c9c17918f9425baf5cd0f1fc1150d1, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:31.842474 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:31.907886 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1434/2854] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:31 2022] PUT /identity/v3/projects/a68bc982b9fe4557bde0474ebc4eb513/users/02c9c17918f9425baf5cd0f1fc1150d1/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:31.973575 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:31.973862 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:31.974138 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.002826 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] MFA Rules not processed for user `02c9c17918f9425baf5cd0f1fc1150d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:32.004256 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:32.067663 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1421/2855] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:32.072088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.078189 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:32.134548 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fe5bc-a302-4cf9-ad93-82ffe58fe156 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.140001 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.140619 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.141073 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.141592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.142587 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.145658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.240876 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1435/2856] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:32 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:32.244837 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.255403 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b202c23-96cf-4f47-a842-4d5dfdf21a25 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.259951 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:32.311202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.311828 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.312116 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.312590 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.313386 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.315683 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.405613 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1422/2857] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:32 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:32.409727 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.420566 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1c93a-376a-41d8-9edb-cd46c9ae63f6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.425361 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.428722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.428722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.428722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.428722 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.429194 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.439978 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1436/2858] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:32.444055 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.453968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca260fd-3a82-4d55-9f77-82deeec9f85f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.458592 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.463202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.463202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.463202 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] PATH_INFO: `/v3/projects/11d38fc90db549a086c86b720377491d/users/9bbb07d0f509454d95a8d5c84cb3cb10/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.474315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d38fc90db549a086c86b720377491d, user_id=9bbb07d0f509454d95a8d5c84cb3cb10, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.476968 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.558936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1423/2859] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:32 2022] PUT /identity/v3/projects/11d38fc90db549a086c86b720377491d/users/9bbb07d0f509454d95a8d5c84cb3cb10/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:15:32.563415 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.573281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61192c83-6178-4d00-bffc-e72905a626ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.629293 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.629890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.630166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.630436 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.631060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.632678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f9d11-4d95-4a38-8299-67714641eaf4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.642498 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1437/2860] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:32.646626 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.657387 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99c600-00ed-4510-80c9-84e8a3c8be45 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.661926 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.662562 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.662843 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.663118 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] PATH_INFO: `/v3/projects/11d38fc90db549a086c86b720377491d/users/9bbb07d0f509454d95a8d5c84cb3cb10/roles/9ff91fa864bb40e18f65458ec16b6f07` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.665045 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d38fc90db549a086c86b720377491d, user_id=9bbb07d0f509454d95a8d5c84cb3cb10, role_id=9ff91fa864bb40e18f65458ec16b6f07)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.667643 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.677455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1424/2861] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:32 2022] PUT /identity/v3/projects/11d38fc90db549a086c86b720377491d/users/9bbb07d0f509454d95a8d5c84cb3cb10/roles/9ff91fa864bb40e18f65458ec16b6f07 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:32.687139 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.687438 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.687719 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.714704 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] MFA Rules not processed for user `9bbb07d0f509454d95a8d5c84cb3cb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:32.715927 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:32.779601 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:32.783099 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1438/2862] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:32.794117 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:32.795805 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:32.852967 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dbb3b-b83e-4b31-ae87-562747b7283f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:32.856588 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:32.857209 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:32.857488 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:32.857761 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:32.867231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:32.868691 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16a2a3-fe86-437b-9838-1d660afed7a5 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:32.878741 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1425/2863] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:33.405824 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:33.466618 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ff245-1593-471b-8412-dd8821ff5e7f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:33.471040 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:33.471658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:33.471965 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:33.472239 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] PATH_INFO: `/v3/users/02c9c17918f9425baf5cd0f1fc1150d1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:33.473338 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c9c17918f9425baf5cd0f1fc1150d1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:33.475241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:33.692572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c9c17918f9425baf5cd0f1fc1150d1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:33.699392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c9c17918f9425baf5cd0f1fc1150d1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:33.705158 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c9c17918f9425baf5cd0f1fc1150d1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:33.780713 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c9c17918f9425baf5cd0f1fc1150d1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:33.789530 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c9c17918f9425baf5cd0f1fc1150d1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:33.796461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1439/2864] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:33 2022] DELETE /identity/v3/users/02c9c17918f9425baf5cd0f1fc1150d1 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:33.801323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:33.801323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:33.801949 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:33.821294 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:33.822484 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:33.880598 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1426/2865] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:33.890778 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:33.892198 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:33.948505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff6299-e0ce-4468-8e64-bc1f2c02c28d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:33.952501 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:33.953091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:33.953360 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:33.953624 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:33.962365 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:33.963793 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:33.973651 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1440/2866] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:34.378404 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:34.379850 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:34.388349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:34.392378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b853-d3d8-497e-b666-88a60b905015 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:34.398378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:34.398378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:34.398378 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:34.398814 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] PATH_INFO: `/v3/projects/a68bc982b9fe4557bde0474ebc4eb513` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:34.399995 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68bc982b9fe4557bde0474ebc4eb513)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:34.401936 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:34.517730 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68bc982b9fe4557bde0474ebc4eb513', 'request_id': 'req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.523686 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68bc982b9fe4557bde0474ebc4eb513', 'request_id': 'req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.592873 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68bc982b9fe4557bde0474ebc4eb513', 'request_id': 'req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.603496 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] The token cache is being invalidate because project a68bc982b9fe4557bde0474ebc4eb513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:34.604323 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.604467 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.605909 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1427/2867] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:34 2022] DELETE /identity/v3/projects/a68bc982b9fe4557bde0474ebc4eb513 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:34.610827 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:34.620472 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774309a-c91e-4f3a-b896-3e1bb1759b7f service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:34.637784 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:34.674917 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:34.675544 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:34.675863 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:34.676134 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] PATH_INFO: `/v3/users/9bbb07d0f509454d95a8d5c84cb3cb10` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:34.677243 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbb07d0f509454d95a8d5c84cb3cb10)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:34.679145 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:34.851081 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbb07d0f509454d95a8d5c84cb3cb10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.851680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbb07d0f509454d95a8d5c84cb3cb10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.864183 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbb07d0f509454d95a8d5c84cb3cb10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.946211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbb07d0f509454d95a8d5c84cb3cb10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.954060 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d3b4d55-2e18-4648-a07a-faa8e274baf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbb07d0f509454d95a8d5c84cb3cb10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:34.960162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1441/2868] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:15:34 2022] DELETE /identity/v3/users/9bbb07d0f509454d95a8d5c84cb3cb10 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:35.366295 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:35.367708 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:35.381231 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb0d26-cd1d-4c44-8895-ee2d2ec24712 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:35.438048 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:35.438749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:35.439021 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:35.439312 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] PATH_INFO: `/v3/projects/11d38fc90db549a086c86b720377491d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:35.440627 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d38fc90db549a086c86b720377491d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:35.442550 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:35.553608 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d38fc90db549a086c86b720377491d', 'request_id': 'req-19e3be4f-343d-4d51-90f3-93f919a80d82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:35.559697 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d38fc90db549a086c86b720377491d', 'request_id': 'req-19e3be4f-343d-4d51-90f3-93f919a80d82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:35.644282 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d38fc90db549a086c86b720377491d', 'request_id': 'req-19e3be4f-343d-4d51-90f3-93f919a80d82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:35.655042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] The token cache is being invalidate because project 11d38fc90db549a086c86b720377491d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:15:35.659633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:35.659633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-19e3be4f-343d-4d51-90f3-93f919a80d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:15:35.659633 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1428/2869] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:15:35 2022] DELETE /identity/v3/projects/11d38fc90db549a086c86b720377491d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:35.663270 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:35.663743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:35.663873 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:35.683344 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:35.684581 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:35.742315 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1442/2870] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:35.751670 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:35.751994 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:35.752289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:35.772409 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:35.773597 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:35.781887 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1429/2871] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:35.786063 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:35.796421 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2be32b-e4c8-4c0c-9cfc-c5afa5bb1f7b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:35.800970 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:35.801570 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:35.801846 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:35.802110 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:35.802747 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:15:35.803049 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:35.804693 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:35.817578 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1443/2872] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:15:35.821953 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:35.833600 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825ea8b-70be-4099-ad90-989edddf8f15 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:35.838455 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:35.839046 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:35.839319 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:35.839587 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:35.840259 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:35.842150 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:35.851604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:35.931512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1430/2873] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:15:35 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:35.935649 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:35.946281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b312-a8b1-42f1-ac4f-42a1ef52f730 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:35.972144 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:36.001525 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:36.002122 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.002404 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.002678 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.003305 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:36.005241 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:36.151707 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1444/2874] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:15:35 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:15:36.157062 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:36.167449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3701d-1c42-4662-ba53-2897abb8a530 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:36.172101 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:36.172726 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.173008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.173281 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.173911 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:36.175512 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:36.185462 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1431/2875] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:36.189362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:36.199439 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67183ed-8ba4-4716-aa81-5d20e41bb787 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:36.204673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:36.205259 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.205536 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.205808 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.206441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:36.208057 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bda05-5474-4ea6-ae10-833990cd6ec4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:36.219149 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1445/2876] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:15:36.223074 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:36.232867 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a138f-141a-4aa3-924c-090a9024af5b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:36.237641 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:36.238273 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.238551 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.238822 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] PATH_INFO: `/v3/projects/5d1ef3cc26ea4e48a4ae9dde3ecc432f/users/76fc895411b74b7aa87e76d09b27fd0e/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.253019 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1ef3cc26ea4e48a4ae9dde3ecc432f, user_id=76fc895411b74b7aa87e76d09b27fd0e, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:36.255644 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:36.313785 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1432/2877] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:15:36 2022] PUT /identity/v3/projects/5d1ef3cc26ea4e48a4ae9dde3ecc432f/users/76fc895411b74b7aa87e76d09b27fd0e/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:15:36.323998 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.324286 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.324561 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.352033 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] MFA Rules not processed for user `76fc895411b74b7aa87e76d09b27fd0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:15:36.353242 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:36.416282 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1446/2878] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:15:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:15:36.427898 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:15:36.429639 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:15:36.485749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8812fad-cdf1-4c33-8674-a6146d75dfc2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:15:36.489244 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a947c2e6ba544767897eac257c1fad7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:15:36.489838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:15:36.490119 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:15:36.490392 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:15:36.498493 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:15:36.500535 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb188f-0581-47e5-94fe-eb55d0508f20 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:15:36.509354 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1433/2879] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:15:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:15:46.413279 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:15:46.505340 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:15.856903 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:15.921128 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc3e7b-4f86-46ba-8d97-8a03bebf4a1e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:15.926427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:15.927575 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:15.928088 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:15.928599 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] PATH_INFO: `/v3/users/76fc895411b74b7aa87e76d09b27fd0e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:15.929953 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76fc895411b74b7aa87e76d09b27fd0e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:15.932137 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:16.126012 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76fc895411b74b7aa87e76d09b27fd0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:16.126921 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76fc895411b74b7aa87e76d09b27fd0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:16.142943 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76fc895411b74b7aa87e76d09b27fd0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:16.221284 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76fc895411b74b7aa87e76d09b27fd0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:16.230321 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76fc895411b74b7aa87e76d09b27fd0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:16.237035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1447/2880] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:16:15 2022] DELETE /identity/v3/users/76fc895411b74b7aa87e76d09b27fd0e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:16.241235 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:16.241740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:16.241871 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:16.266042 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:16:16.268106 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:16.344362 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1434/2881] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:16:16.356981 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:16.360475 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:16.427822 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ecd51-c0e2-448e-bdfd-4788a869b8b9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:16.433461 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:16.434102 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:16.434389 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:16.434701 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:16.457960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:16.457960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384602f-b2e6-4a35-8ee4-3478b5884cb3 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:16.457960 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1448/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:16:16.930288 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:16.931845 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:16.951651 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd337e-cd23-41aa-94cf-e4a59b7a44b6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:16.957331 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:16.958043 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:16.958324 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:16.958602 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] PATH_INFO: `/v3/projects/5d1ef3cc26ea4e48a4ae9dde3ecc432f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:16.959736 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1ef3cc26ea4e48a4ae9dde3ecc432f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:16.961838 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.089824 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1ef3cc26ea4e48a4ae9dde3ecc432f', 'request_id': 'req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:17.096662 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1ef3cc26ea4e48a4ae9dde3ecc432f', 'request_id': 'req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:17.187868 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1ef3cc26ea4e48a4ae9dde3ecc432f', 'request_id': 'req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:17.198696 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] The token cache is being invalidate because project 5d1ef3cc26ea4e48a4ae9dde3ecc432f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:16:17.202740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:17.202740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5a0f1c2-6f5d-48ea-a5e0-bdc505e78a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:17.202740 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1435/2883] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:16:16 2022] DELETE /identity/v3/projects/5d1ef3cc26ea4e48a4ae9dde3ecc432f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:17.206429 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.206913 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.207035 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.228223 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:16:17.229473 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:17.246392 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:17.287838 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1449/2884] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:16:17.296775 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.297289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.297289 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.317491 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:16:17.318682 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:17.326547 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1436/2885] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:16:17.331297 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.340881 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3f2d5-d741-4358-967b-5b442d350f69 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.345658 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.346396 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.346671 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.346941 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.347572 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 11:16:17.347905 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.349468 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.361456 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1450/2886] 10.222.0.48 () {58 vars in 1153 bytes} [Thu Aug 11 11:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 11:16:17.365508 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.374882 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18ed2-61e8-4e2b-91a4-b973309ab072 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.380024 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.380605 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.380883 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.381154 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.381889 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.383780 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.470008 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1437/2887] 10.222.0.48 () {60 vars in 1153 bytes} [Thu Aug 11 11:16:17 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:16:17.474584 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.486775 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16e825-65da-4297-adb8-0ac262a8d1c8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.546309 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.546930 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.547335 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.547613 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.548356 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.550300 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.651487 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1451/2888] 10.222.0.48 () {60 vars in 1144 bytes} [Thu Aug 11 11:16:17 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 11:16:17.655583 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.667803 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9369-513a-47de-a13d-b7ef9d8ad954 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.670900 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.671497 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.671808 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.672087 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.672727 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.674857 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.685179 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1438/2889] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:16:17.689080 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.699809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc8ec-2ec3-43c8-a65a-89e091fef79c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.704492 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.705113 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.705505 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.705780 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.706414 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.708091 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cda27-99c8-4cbc-a069-fab8b88c2dbe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.718964 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1452/2890] 10.222.0.48 () {58 vars in 1122 bytes} [Thu Aug 11 11:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 11:16:17.723035 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.733449 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0607c-5dd4-4182-8d6c-44dc9cfe54dc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.740532 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.741315 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.741666 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.742038 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] PATH_INFO: `/v3/projects/d9ee878e701e45c18d2f3181369a0a1d/users/33c850bf6c1749e396f5b0c018a99b78/roles/933643980c98414dad5713dc8820df36` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.758063 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ee878e701e45c18d2f3181369a0a1d, user_id=33c850bf6c1749e396f5b0c018a99b78, role_id=933643980c98414dad5713dc8820df36)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:17.760763 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:17.817531 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1439/2891] 10.222.0.48 () {60 vars in 1483 bytes} [Thu Aug 11 11:16:17 2022] PUT /identity/v3/projects/d9ee878e701e45c18d2f3181369a0a1d/users/33c850bf6c1749e396f5b0c018a99b78/roles/933643980c98414dad5713dc8820df36 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 11:16:17.827743 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.828120 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.828395 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:17.855355 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] MFA Rules not processed for user `33c850bf6c1749e396f5b0c018a99b78`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:16:17.856592 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:17.921518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1453/2892] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 11:16:17.932749 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:17.934416 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:17.956918 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:17.992460 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab24aa-bfc6-4b24-8890-ef95e6c96ce0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:17.996991 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1e9a4ceebcc44389144522cec3cc678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:17.997585 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:17.997859 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:17.998127 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:18.006819 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:18.008272 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:18.018250 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1440/2893] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:16:27.918512 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:28.056615 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=17 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:37.282289 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:37.283735 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:37.338976 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13611-70da-4a95-9017-5420644e0a03 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:37.343078 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:37.343680 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:37.343987 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:37.344262 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:37.345952 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:37.406549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:37.408023 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2f5a2-4bec-48a4-b150-15e1869dd881 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:37.417507 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1454/2894] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:16:38.874485 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:38.885506 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cd47a-26cb-4029-99d6-e8ea489c9ba5 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:38.890852 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:38.891474 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:38.891752 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:38.892060 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] PATH_INFO: `/v3/users/33c850bf6c1749e396f5b0c018a99b78` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:38.893384 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c850bf6c1749e396f5b0c018a99b78)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:38.895296 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:39.111020 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c850bf6c1749e396f5b0c018a99b78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:39.111681 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c850bf6c1749e396f5b0c018a99b78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:39.120349 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c850bf6c1749e396f5b0c018a99b78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:39.135242 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c850bf6c1749e396f5b0c018a99b78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:39.207499 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c850bf6c1749e396f5b0c018a99b78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:39.215405 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1441/2895] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:16:38 2022] DELETE /identity/v3/users/33c850bf6c1749e396f5b0c018a99b78 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:39.220132 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:39.220650 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:39.220782 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:39.242316 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] MFA Rules not processed for user `697f0ea802614a91b28dfb28d2104fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 11:16:39.243673 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:39.316117 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1455/2896] 10.222.0.48 () {58 vars in 958 bytes} [Thu Aug 11 11:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 11:16:39.322758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:39.324799 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:39.391072 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a3728-c6a5-4d94-8b8a-292e34e15d2f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:39.395391 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ab7cc322d04f84ab56c54cf8429596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:39.396216 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:39.396492 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:39.396760 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:39.406302 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:39.407729 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:39.417776 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1442/2897] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:16:39.887789 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:39.889264 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:39.901890 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d378d1f-a4fc-40c4-823e-bfb9302cf811 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:39.906532 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:39.907166 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:39.907594 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:39.907900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] PATH_INFO: `/v3/projects/d9ee878e701e45c18d2f3181369a0a1d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:39.910763 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ee878e701e45c18d2f3181369a0a1d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:39.912791 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:40.059417 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:40.062161 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ee878e701e45c18d2f3181369a0a1d', 'request_id': 'req-130ce8d7-4a86-4aed-9e83-56e55243a37f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:40.068221 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ee878e701e45c18d2f3181369a0a1d', 'request_id': 'req-130ce8d7-4a86-4aed-9e83-56e55243a37f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:40.196362 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ee878e701e45c18d2f3181369a0a1d', 'request_id': 'req-130ce8d7-4a86-4aed-9e83-56e55243a37f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:40.210138 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] The token cache is being invalidate because project d9ee878e701e45c18d2f3181369a0a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:16:40.210898 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:40.211605 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:40.213786 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1456/2898] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:16:39 2022] DELETE /identity/v3/projects/d9ee878e701e45c18d2f3181369a0a1d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:40.985194 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:40.987756 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:41.060978 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f95d2-091b-44b1-954b-f328e489e8df service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:41.073067 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:41.123274 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:41.123839 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:41.124122 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:41.124393 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:41.126104 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:41.137543 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:41.139134 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:41.150604 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1443/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:16:44.881428 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:44.883140 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:44.952029 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ce8d7-4a86-4aed-9e83-56e55243a37f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:44.957254 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:44.957888 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:44.958162 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:44.958480 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] PATH_INFO: `/v3/users/2a9dbe3b7fdd482681b6ab94c26f4e71` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:44.959640 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9dbe3b7fdd482681b6ab94c26f4e71)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:44.961674 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:45.182433 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9dbe3b7fdd482681b6ab94c26f4e71'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.183085 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9dbe3b7fdd482681b6ab94c26f4e71'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.195390 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9dbe3b7fdd482681b6ab94c26f4e71'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.397104 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9dbe3b7fdd482681b6ab94c26f4e71'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.405017 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9dbe3b7fdd482681b6ab94c26f4e71'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.411249 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1457/2900] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:16:44 2022] DELETE /identity/v3/users/2a9dbe3b7fdd482681b6ab94c26f4e71 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:45.669425 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:45.675144 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6775f7-2719-49a6-9ad6-1a1fa165e059 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:45.737876 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:45.738505 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:45.738786 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:45.739059 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] PATH_INFO: `/v3/projects/b1d394882f294c6283721bcc6f8ff04d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:45.740192 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d394882f294c6283721bcc6f8ff04d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:45.744010 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:45.861977 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d394882f294c6283721bcc6f8ff04d', 'request_id': 'req-195b4cd1-ae1a-431d-a000-9efbadaaca88'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.868130 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d394882f294c6283721bcc6f8ff04d', 'request_id': 'req-195b4cd1-ae1a-431d-a000-9efbadaaca88'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.939961 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d394882f294c6283721bcc6f8ff04d', 'request_id': 'req-195b4cd1-ae1a-431d-a000-9efbadaaca88'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.951432 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] The token cache is being invalidate because project b1d394882f294c6283721bcc6f8ff04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:16:45.951710 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.952446 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:45.953996 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1444/2901] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:16:45 2022] DELETE /identity/v3/projects/b1d394882f294c6283721bcc6f8ff04d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:45.957985 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:45.959402 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:45.972518 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8e12-4bbc-4531-8e2e-507c34846edc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:46.019549 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:46.028482 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:46.029105 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:46.029388 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:46.029659 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] PATH_INFO: `/v3/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:46.030805 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfbd7fe60344a4b9f8a751b4ccbc1ef)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:46.032920 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:46.223070 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfbd7fe60344a4b9f8a751b4ccbc1ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.223644 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfbd7fe60344a4b9f8a751b4ccbc1ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.236064 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfbd7fe60344a4b9f8a751b4ccbc1ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.284427 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfbd7fe60344a4b9f8a751b4ccbc1ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.291470 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfbd7fe60344a4b9f8a751b4ccbc1ef'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.297281 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1458/2902] 10.222.0.48 () {58 vars in 1224 bytes} [Thu Aug 11 11:16:45 2022] DELETE /identity/v3/users/ecfbd7fe60344a4b9f8a751b4ccbc1ef => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:46.564866 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:16:46.570504 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:16:46.580160 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4cd1-ae1a-431d-a000-9efbadaaca88 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:16:46.615248 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:46.638549 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697f0ea802614a91b28dfb28d2104fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '345ef69aeca14be9ac484aeebec744ed', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:16:46.639165 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:16:46.639440 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:16:46.639718 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] PATH_INFO: `/v3/projects/346242baa82d43418f6f5f5307adf19b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:16:46.640846 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=346242baa82d43418f6f5f5307adf19b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:16:46.642758 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:16:46.759383 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346242baa82d43418f6f5f5307adf19b', 'request_id': 'req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.765989 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346242baa82d43418f6f5f5307adf19b', 'request_id': 'req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.827359 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346242baa82d43418f6f5f5307adf19b', 'request_id': 'req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.837717 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] The token cache is being invalidate because project 346242baa82d43418f6f5f5307adf19b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 11:16:46.842365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.842365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c0e0f9ca-00e6-4b05-bee9-ed310507c0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 11:16:46.842365 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1445/2903] 10.222.0.48 () {58 vars in 1233 bytes} [Thu Aug 11 11:16:46 2022] DELETE /identity/v3/projects/346242baa82d43418f6f5f5307adf19b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 11:16:56.293809 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:16:56.834768 dv-cinder-820027-17-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 11:18:00.642900 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 11:18:00.644495 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:18:00.712177 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23f7ee-b6de-439a-b194-565bb70b83cc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 11:18:00.770855 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25af3fcbdf174cc2bf22fb99c83575f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '17ebc11defe74b93b10eaac92f9dacd5', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 11:18:00.771483 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 11:18:00.771832 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 11:18:00.772374 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 11:18:00.774096 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 11:18:00.785108 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 11:18:00.786684 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45d229-2d75-4c77-8781-49e3b744640a service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 11:18:00.796441 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1459/2904] 10.222.0.48 () {60 vars in 1365 bytes} [Thu Aug 11 11:18:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 11:18:10.792211 dv-cinder-820027-17-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=58 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}